Archive for the 'system' Category

Linux booting optimizations

Published
by
almlys
on April 11, 2009
in sysadmin, system and tech
. 0 Comments

This is a smal guide that I use myself in new or current Debian GNU/Linux installations, to reduce a bit (about some seconds), the overall booting time.

To start, we need a way to measure the time that our system needs to boot up.

  • Bootchart: It measures time required to boot the system, and displays it in a graphical way.
    • $sudo apt-get install bootchart
    • Edit /etc/grub/menu.lst, and add "init=/sbin/bootchartd" to the kopt line.
    • $sudo update-grub
    • Edit /etc/bootchartd.conf and set "AUTO_STOP_LOGGER" to "yes"

Now some basic "safe" optimizations. Optimizations that you can do without breaking the system.

  • ReadAhead: It preloads all files required to boot at the begining of the boot process. (Note: If you have preload installed it must be dissables when the first run is done.)
    • $sudo apt-get install readahead
    • If preload is installed, temporally dissable it by adding "exit 0" at the beggining to /etc/init.d/preload
    • Enable profile on first run: $sudo touch /etc/readahead/profile-once
    • reboot system (yes you need to reboot, sorry, is one of these exceptions when you need to reboot Linux.)
  • preload: It learns with system libraries are most used, and preloads them on demand.
    • $sudo apt-get install preload
  • dash: By default the OS ships with bash that is extremely slow, thus all init shell scripts run very slow. Dash is a faster shell implementation. I saw an increase of 6 seconds in my experiments.
    • $sudo apt-get install dash
    • Install dash as default sh interpreter: $sudo dpkg-reconfigure dash
  • You need an e-amail server?: By default the OS will ship with a full e-mail server, because some desktop applications an other stuff depend on it. Debian comes with Exim, the light version by default. This means that you will have an extra useless process consuming resources at boot time, and during execution. You can't remove it because it's needed, but you can replace it with a null mailer implementation. So there are two possible packages avaliable depending on your e-mail needs. Of course if you really need a full e-mail server in you computer, then you may continue with wathever you have.
    So, we have ssmtp and esmtp-run

    • ssmtp: This mailer will deliver all mail to a remote e-mail server, it does not do local mail delivery it's only usefull when you already have an e-mail server in your network.
      • $sudo apt-get install ssmtp
      • Configure your remote e-mail server. $sudo dpkg-reconfigure ssmtp
    • esmtp: Same as ssmtp, but does local mail delibery. If you don't have a remote mail server to use, just use localhost as a mail server, it will try to deliver mail to it, but since there is no service running it will fail, thus it will only do local delibery. This is perfect for a laptop where I'm only interesting on recieving locally system generated mails, I don't want my laptop to send e-mails outside. (In fact most Linux distros ship by default with an e-mail server that will do remote delivery by default, altough it's only listenning on localhost, I see that as a possible security threat that any user could exploit, altough nowadays most e-mail servers will reject those e-mails.).
      • $sudo apt-get install esmtp-run
  • quiet saves time also. The "quiet" kernel parameter it's usually enabled on most distros, but if it is not enabled in your distro, it's poosible to save some time. Just check that "quiet" is present in "defoptions" of your /boot/grub/menu.lst configuration file.
  • remove some useless garbage: Some completely useless services, that only waste ram, and time. (Always check when you remove something from your system, that you are not deinstalling your entire operating system due to dependencies. I at least know a case of someone that wanted to unistall mysql but unistalled the entire os, because the os had dependencies over mysql?? (This only happens on Ubuntu, Debian does not have this problem ;-P )
    • Who needs "inetd", I don't use anything that it provides. $sudo apt-get --purge remove openbsd-inetd
    • WTF is avahi? More useless garbage. I remove it as soon as I install a new system. $sudo apt-get --purge remove avahi-daemon avahi-autoipd
    • localepurge: Why do you need your os avaliable in 560 and more languages? Them waste disk space.
      • $sudo apt-get install localepurge
      • Configure it to the locales that you want to keep $sudo dpkg-reconfigure localepurge
      • And free about 1Gigabyte or more of space. $sudo localepurge
    • orphaner: Remove unused orphaned system libraries.
      • $sudo apt-get install deborphan
      • Remove all unused garbage: $sudo orphaner
    • autoremove unused stuff: Remove all those packages that were installed but are currently not used and we don't want them in our system.
      • $sudo apt-get --purge autoremove
    • remove unused config files: Unistalled packages will drop a lot of garbage in the /etc directory, so remove it.
      • $sudo dpkg --purge `dpkg --get-selections | grep deinstall | cut -f1`

Some dangerous optimizations. These optimizations may break your system into an unboatable status, and also require lot of user intervention to configure.

  • insserv: Reorders the init scripts. May break some thing that need to be manually fixed.
  • runit: Another init system, requires a lot of time to configure and get a boatable, system. NOT recomended for currently installed systems. Should be implemented on new systems, with lot of patience. I personally do not have enough patience and/or time to configure runit.

That's all for today.

mod_rpaf

Published
by
almlys
on December 7, 2007
in english, sysadmin and system
. 1 Comment

When you are using an inverse proxy, to multiplex several apache and other webservers through the same ip address, all your bad written and bad designed applications will start to log and use the proxy ip address instead of the real one. Your "allow/deny from ip_address" directives will stop working, etc...

There is a very very dirty hack for php applications, using the auto_prepend_file directive in the php ini file, that allows you to swap the proxy ip address with the client one, so some applications such as wordpress can continue logging the real ip address instead of the proxy one.

 
<?php
        function xinverse() {
                if($_SERVER["REMOTE_ADDR"]=="172.26.0.27" &amp;&amp; !empty($_SERVER["HTTP_X_FORWARDED_FOR"])) {
                        list($_SERVER["REMOTE_ADDR"])=split(",",$_SERVER["HTTP_X_FORWARDED_FOR"]);
                }
        }
 
        xinverse();
?>
 

The problem is that this very dirty hack does not work with phpbb. Also, you need to do similar hacks for other applications written in other languages like python, perl, etc... And there is no way to use the apache ip based directives.
The solution, a module that unfortunately is not available in debian stable (only in sid and is not usable for stable). The mod_rpaf, checks if the request comes from your controlled remote proxy, and if that is the case it grabs the first ip address from the HTTP_X_FORWARDED_FOR tuple and rewrites REMOTE_ADDR with it.
This module is really really simple to compile, thanks to apxs and now after compiling it in a testing virtual domain (thanks xen) I have installed it in the production virtual domain (here).

Update: allow/deny form does not work with mod_rpaf as I expected, you need to perform those validations in the proxy side.

SetEnvIf X-Forwarded-For ^172\.26\.0\.17 let_me_in
 
Order allow,deny
allow from env=let_me_in
ErrorDocument 403 /isdown.php

It's dirty, but it works. Be carefull here, we can only do this if we have our trusted proxy in front of the server, if we open the access then anyone will bypass by setting any arbitrary x-forwarded-for header.