memes that damage debian

The Debian project is afflicted with several damaging memes. One that is causing problems at the moment is the idea that life should be fair. Unfortunately life is inherently unfair, it's not fair that those of us who were born in first-world countries (the majority of Debian developers) have so many more opportunities than most people who are born in "developing" countries, and things just continue to be unfair as you go through life. Unfair things will happen to you, deal with it and do what is necessary to have a productive life!

When one developer has regular long-term disputes with many other developers the conclusion is that the one person who can't get along is at fault. We can debate about whether one or two significant disputes are a criteria for this or whether having a dozen developers disagreeing with them is the final point. But the fact is that if there is a large group of people who work together well and an individual who can't work with any of them then there is only one realistic option, the individual needs to go find some people that they can work with - they can resign or be expelled. The fact that something slightly unfair might have happened a year ago is no reason for pandering to an anti-social developer. The fact that expelling a developer for being anti-social is unfair to them is no reason for damaging the productivity of all Debian developers.

Another problemmatic meme is the idea that we have to tolerate everyone - even those who are intolerant (known as the Limp Liberal meme). When someone has no tolerance for others (EG being racist or practicing sexual discrimination) then they have no place in a community such as Debian. They need to be removed sooner rather than later. All Debian developers know the problems caused by deferring such expulsion.

The final damaging meme that I have observed is you can't force a volunteer to do any work. On it's own that statement is OK, but the interpretation commonly used in Debian is you can't take their job away from them either. The most common example of this is when a developer is not maintaining a package and someone else does an NMU (non-maintainer upload) to fix a bug, the developer then flames the person who did this (usually to fix a severe bug). It seems to be believed that a Debian developer owns their packages and has a right to prevent other people from working on them. This attitude also extends to all other aspects of Debian, there are many positions of responsibility in Debian that are not being adequately performed and for which volunteers are offering to help out but being refused.

The idea of the GPL is that when a program is not being developed adequately it can be taken over by another person. However when that program is in a Debian package the developer who owns it can refuse to allow this.

more about vista security

While reading the discussion of Vista security on Bruce Schneier's blog it occurred to me that comparing the issues of DRM that face MS with the issues faced by SE Linux developers provides some benefits.

SE Linux is designed to enable the owner of a computer to effectively enforce security policies to protect their system integrity and the confidentiality of their data. Some of the SE Linux users (military users) use all the available features, but most people only use the targeted policy which provides a sub-set of the system integrity and data confidentiality protections that are available to give a greater ease of use.

Fedora and Red Hat Enterprise Linux ship with the SE Linux targeted policy enabled by default. This policy is not something that most users notice. The main program that is commonly used which might have an issue with the default SE Linux policy is Apache. Fedora and RHEL systems that do not run Apache (which is most of them) can run SE Linux with almost no costs.

setsebool -P httpd_disable_trans 1
/etc/init.d/httpd restart

It seems clear to me that there is no good reason for disabling SE Linux by default. There are reasons for running a particular daemon in the unconfined_t domain via the FOO_disable_trans boolean. EG to run Apache without restrictions you would type the above commands.

In spite of the SE Linux targeted policy being so easy to use, the fact that it prevents certain daemon compromises from allowing immediate control of the system, and also prevents some kernel exploits from working, there are still some people who turn it off when installing Fedora and RHEL systems and advise others to do so.

Given this great fear of some people to use a technology that is specifically designed for their own benefit I find it difficult to imagine that any users will be inclined to accept the MS DRM technology that is specifically designed to go against their interests.

ESR claims that the 64bit transition is the critical period for Linux to move on the desktop. While he makes many interesting observations I don't find his argument convincing. Firstly current P4 CPUs with PAE can handle significantly more than 4G of RAM (64G is possible now). Secondly it's quite possible to run a 64bit kernel with 32bit applications, this means that you can use a 64bit kernel to access >64G of RAM with each 32bit application getting direct access to something slightly less than 4G of virtual memory. As ESR's point seems based on total system memory the 4G per application doesn't seem to be an issue. As an aside the only applications I've used which could benefit from >4G are database servers (in the broadest interpretation - consider an LDAP server as a database) and JVM environments. Again it would not be difficult to ship an OS with mostly 32bit code that has a 64bit kernel, JVM, and database servers.

I hope that Vista will be a good opportunity for mass transition to Linux. Vista offers little that users desire, many things that will hurt them, and is expensive too!

With Vista you pay more for the OS and more for the hardware (Vista has the usual excessive Windows hardware requirements plus the extra hardware for TPM encryption) without providing anything substantial in return.

What I want to see next is support for Security Enhanced X to protect desktop environments against hostile X clients. This will make a significant difference to the security of Linux desktop environments and provide another significant benefit for choosing Linux over Windows. While MS is spending their effort in making their OS act against the best interests of the users we will keep making Linux enforce the access controls that users need to protect their systems. Hopefully Linux users will choose to use SE-X, but if they don't they are given the freedom to make that choice - unlike the poor sods who use Windows.

email disclaimers

Andre Pang blogs about the annoyance of email disclaimers. For a while I had a .sig indicating that it was a condition of sending email to me that the sender agrees to legalistic terms in their .sig being inapplicable to me.

220 smtp.sws.net.au ESMTP Postfix - by sending email to this server you agree that any legalistic sig in your message does not apply to anyone who receives the message through this service.

Now I have changed my Postfix greeting to the above. Anyone who sends me mail agrees that their .sig does not apply to me. Suggestions for improvements to the above text are welcome.

source dump blog

Inspired by Julien Goodwin's post I created a new blog for myself named Source Dump. Source is different to other blog content in that updates to fix bugs may be required (generally I believe that ideally blog posts should not be edited once published and in the rare cases of editing being necessary all such edits should be appended to the end), and in that it may be longer than is suitable for a Planet feed. Finally Planet and other aggregators may mess up source code. If the source is only visible through my blog then I can be reasonably sure that it is usable by everyone who sees it.

So when I have source code to publish that is related to blog postings or mailing list postings I now have a place for it (I haven't yet made a posting).

I will not be submitting my Source Dump blog for syndication anywhere, but if anyone wants to syndicate it they are welcome to do so. I don't desire that it not be syndicated, I merely recommend that it not be syndicated for the convenience of readers due to the potential large size of postings and the potential for postings to be broken by aggregation.

music for children

Adam Rosi-Kessel made an interesting post about They Might Be Giants producing children's music because their original fan base are now old enough to have children.

From casual inspection of the crowds at events such as Linux Conf AU it seems to me that many serious Linux people are also at the right age to have young children, and several blogs that are syndicated on Linux Planets provide evidence of this. Therefore it seems that there is a market for Linux related children's music.

Many aspiring artists complain about the difficulty of establishing a reputation. I think that if someone was to release OGG and FLAC recordings of a children's version of the Free Software Song under a Creative Commons license then they would get some immediate publicity through the blog space and Linux conferences which could then be used to drive commercial sales of children's music.

While on the topic, it would be good to have a set of children's songs and nursery rhymes to teach children from a young age about the community standards that we share in the Free Software community. There is no shortage of propaganda that opposes our community standards, the idea that sharing all music and software is a crime is being widely promoted to children.

google reader

From a suggestion on my previous blog entry I decided to test out google reader.

The first problem was that it caused Konqueror to SEGV in etch, I filed a bug report and switched to Firefox.

Next to add my feeds I had to either export them in OPML format or add them one at a time, there is no support for pasting in a list of URLs. If I was writing a RSS syndication program I would also make it parse the config files of some of the common programs, parsing a Planet config file is pretty easy.

I added a feed for a friend who's server seems to be down. While doing so I tried to add another feed, the google reader accepted the command to add the second feed but didn't actually do so - it was fortunate that I was pasting it in not typing it...

The killer issue is that it seems to be impossible to merge feeds. I want to read both Planet Linux Australia and Planet Debian, there are some people who are on both planets (EG me). So it makes no sense to do anything other than display both of them in the same view.

At this time it seems that google reader is unsuitable for my use. However it is a fairly slick system and I imagine that it would work quite well for people who have different needs to me. If you want to read the blogs of a few friends then it probably works really well. It just seems not to work well for a set of meshed communities (Debian developers and Linux users in Australia for example).


Please let me know if I somehow missed some configuration options to make google reader do what I want.

planet - resource use

I just noticed that /usr/bin/planetplanet is using about 120M of RAM. This isn't currently a problem as I'm running it on a machine with 256M of RAM, however I would like to run my web server on a 96M Xen instance. 120M for planetplanet is probably going to cause bad performance on a web server with 96M of physical RAM allocated.

This is a serious problem for me as the Xen server in question can't be upgraded any more (the motherboard has as much RAM as it can handle).

Are there any other free syndication programs that use less memory?

Are there any good free syndication services that I could use instead of running my own planet?

DOSing Windows Vista

Chris Samual writes a good summary of Peter Gutmann's analysis of the cost of Vista (in terms of DRM).

The following paragraph in the article however seemed more interesting to me:
Once a weakness is found in a particular driver or device, that driver will have its signature revoked by Microsoft, which means that it will cease to function (details on this are a bit vague here, presumably some minimum functionality like generic 640x480 VGA support will still be available in order for the system to boot). This means that a report of a compromise of a particular driver or device will cause all support for that device worldwide to be turned off until a fix can be found.

Now just imagine that you want to cause widespread disruption - a DOS (Denial Of Service) attack against Windows users. What better option than to cause most of them to have hardware that's not acceptable to the OS? I expect that there will be many instances of security holes in drivers and hardware being concealed by MS because they can't afford the PR problems resulting from making millions of machines cease functioning. But just imagine that someone finds hardware vulnerabilities in a couple of common graphics drivers or pieces of hardware and publicly releases exploits shortly before a major holiday. If it's public enough (EG posted to a few mailing lists and faxed to some major newspapers) then MS would be forced to invoke the DRM measures or lose face in a significant way. Just imagine the results of stopping 1/3 of machines working just before Christmas!

Of course after the first few incidents people will learn. It shouldn't be difficult to configure a firewall to prevent all access to MS servers so that they can't revoke access, after all our PCs are important enough to us that we don't want some jerk in Redmond just turning them off. Of course disabling connections to MS would also disable security updates - but we all know that usability is more important than security to the vast majority of users (witness the number of people who happily keep using a machine that they know to be infected with a virus or trojan).

If this happens and firewalling MS servers becomes a common action, I wonder if MS will attempt the typical malware techniques of using servers in other countries with random port numbers to get past firewalls. Maybe Windows updates could be spread virally between PCs, this method would allow infecting machines that aren't connected to the net via laptops.

Finally, I recommend that people who are interested in such things read Eastern Standard Tribe by Cory Doctorow, he has some interesting ideas about people DOSing corporations that they work for which seem surprisingly similar to what MS is doing to itself. I'll post about my own observations of corporations DOSing themselves in the near future.

MythTV

I've just been trying to set up a MythTV system, I've had the hardware for this for a while but until my TV broke I hadn't found the time to work on it.

My planned hardware was a P3 system for both frontend and backend. However the Debian MythTV packages take over 250M of memory for each of the frontend and backend programs (more than 500M for Myth programs alone), combine that with some memory use for MySQL, for the X server, and for the rest of a functional Linux system and a machine with 512M of RAM does not perform well. Unfortunately it seems that most (if not all) desktop P3 systems only support 512M of RAM (at least none of my collection of P3 machines does). P3 server systems support considerably more RAM (I used to have a SMP P3 server with 1G of RAM), but a server class machine is not what you want in your lounge room. P4 systems support more RAM but also take more than twice the electricity of P3 systems. The extra electricity use is a waste for a machine that will probably run 24*7, and also requires extra cooling (and therefore probably louder fans which is bad for a machine used for playing audio). Given that I desire the capabilities of a P4 in terms of RAM support and electricity use that is not excessive, my options are one of the recent Intel Core CPUs or AMD64 (both of which apparently use less electricity - not sure whether they would use as little as a P3) or a CPU designed for low power use (maybe a Via). Either option costs more than I prefer to spend (I want to use a second-hand machine valued at $129).

Of course if I could get Myth to use less memory then that would allow me to run both parts of MythTV on the one machine. Please offer any advice you can think of in regard to minimising memory use for Myth on Debian.

Given that I am unlikely to be able to optimise memory use as much as I desire it seems that I will have to go to a split model and try using an existing Pentium-D server machine as the backend and the P3 as a frontend. The up-side of this is that I can turn off the frontend machine when it's not being used (the server runs 24*7 anyway). I will need to get more RAM for the Pentium-D machine, but I had wanted to do that anyway.

The next thing I will have to do is to write SE Linux policy for MythTV and find out why /usr/lib/libmp3lame.so.0.0.0 needs both execmod and execstack (and either fix the library or write policy to permit that accss). I'm happy to have a MythTV server program running on a stand-alone machine with unrestricted access to all resources, but when it's going to run on a machine that is more important to me I need to lock it down.

Now to the design of MythTV. It's unfortunate that they don't support better multitasking options. One feature I would like to see is the ability to play MP3s in the background and then have the music pause whenever TV is selected. When I watch a live show (such as the news) I would like to listen to MP3s before it starts and then at the end of the show (and maybe during commercial breaks) have the MP3 playing resume where it left off. This is exactly the traditional lounge-room functionality we are all used to, when a commercial starts you mute the TV and un-pause the CD player! The advantage of MythTV is that you could do both with a single button - or even automatically with advert recognition!

The method of selecting MP3 files to play is also a little cumbersome and the option to add a song after the current one in the play list doesn't seem to work. Also it's a pity that there is no convenient option to sort the music list by artist, genre, or sone name - there are grouping options but they are all separate and I would like to change the sorting via a single key press.

The startup sequence of MythTV regenerates some images (which takes a few seconds), it seems that this is something that could be cached for a faster startup. Also when I add new MP3 files to the store I have to manually request a re-scan. It's a pity that it can't just check the size and time-stamp of the directories which contain MP3 files and do an automatic re-scan if they change.

installing Debian Etch

A few days ago I installed Debian/Etch on my Thinkpad. One of the reasons for converting from Fedora to Debian is that I need to run Xen and Fedora doesn't support non-PAE machines with Xen. Ironically it's hardware supplied to me by Red Hat (Thinkpad T41p) that is lacks PAE support and forces me to switch to Debian. I thought about just buying a new dual-core 64bit laptop, but that seems a bit extravagant as my current machine works well for everything else.

Feeling adventurous I decided to use the graphical mode of the installer. I found it a little confusing, at each stage you can double-click on an item or click on the continue button to cause the action to be performed. The partitioning section was a little unclear too, but given that it has more features than any other partitioning system I've seen I wasn't too worried (options of creating a degraded RAID array and for inserting a LUKS encryption layer at any level are really nice). The option to take a screen-shot at any time was also a handy feature (I haven't yet inspected the PNG files to see what they look like).

Another nice feature was the way that the GUI restarts after a crash. While it was annoying that the GUI started crashing on me (and would have prevented a less experienced user from completing the install) the fact that it didn't entirely abort meant that I could work around the problem.

I have not yet filed any bug reports against the installer because I have not done a repeatable install (there is a limit to how much testing I will do on my most important machine). In the next few days I plan to do a few tests of the graphical installer on test hardware for the operations that are important to me and file appropriate bug reports. I encourage others to do the same, the graphical mode of the installer and the new encryption and RAID features are significant improvements to Debian and we want them to work well.

I have realised that it won't be possible to get SE Linux as good as I desire before the Etch release, even if the release is delayed again. I'm not sure how many fixes can go in after the release (I hope that we could move to a model similar to RHEL - but doubt that it will happen). So I now plan to maintain my own repository of Etch SE Linux packages and for other packages which need changes to make them work in the best possible manner with SE Linux. I will append something like ".se1" to the version of the packages in question, this means that they will be replaced if a security update is released for the official package. Apart from the SE Linux policy packages (for which any security updates will surely involve me) the changes I am going to make will not be major and will be of less importance than a security update.

I will also add other modified and new packages to my repository that increase the general security of Etch. Apart from SE Linux all the changes I intend to host will be minimal cost issues (IE they won't break things or increase the difficulty of sys-admin tasks), and the SE Linux related changes will not break anything on non-SE systems. So someone who wants general security improvements without using SE Linux might still find my repository useful.

another visual migraine

This morning while travelling to work by tram I had another visual migraine. It was a little worse than last time, not only did everything I focussed on appear to shimmer, but things went a bit grey at my peripheral vision. I had a headache as well although it was very mild (not the typical migraine headache).

It was convenient that the vision problems almost exactly matched the time of my tram journey so that it didn't cause me to waste much time. One visual migraine every three months is something that won't inconvenience me much. I just hope that I don't get other migraine symptoms in future.

encryption speed - Debian vs Fedora

I'm in the process of converting my Fedora/rawhide laptop to Debian.

On Fedora the AES encrypted filesystems deliver about 38MB/s read speed according to dd. On Debian the speed is 2.4MB/s when running Xen and 2.7MB/s when not running Xen. The tests were done on the same block device.

Debian uses a SMP kernel (there are no non-SMP kernels in Debian), but I don't expect this to give an order of magnitude performance drop. Both systems use i686 optimised kernels.

Update: As suggested I replaced the aes module with the aes_586 module. Unfortunately it made no apparent difference.

Update2: As suggested by a comment I checked the drive settings with hdparm and discovered that my hard drive was not using DMA. After I configured the initramfs to load the piix driver first it all started working correctly. Thanks for all the suggestions, I'll post some benchmarks of encryption performance in a future blog entry.

hybrid Lexus is best luxury car

The Lexus GS 450 hybrid petrol/electric car has been given the award for Australia's best luxury car!

The judging for this contest rated fuel efficiency as low importance, because luxury car owners traditionally aren't very concerned about such things. The Lexus won because of it's quiet engine (can't beat an electric motor at low speed), high performance (3.5L petrol engine that outperforms mode 4L engines because of the electric motor assistance), safety, security, and other factors.

There has been an idea that hybrid cars are only for people who want to protect the environment at all costs. The result of this contest proves that idea to be false. The Lexus won by simply being a better luxury car, the features that benefit the environment also give a smoother and quieter ride and higher performance - which are factors that are very important to that market segment! Also it wasn't even a close contest, the nearest rival achieved an aggregate score of 9% less (a significant difference as there was a mere 2.5% difference in score between the 2nd place and 5th place).

This of course shouldn't be any surprise. The high torque that electric motors can provide at low speed is well known - it's the reason for Diesel-electric hybrid power systems in locomotives. It was only a matter of time before similar technology was introduced for cars for exactly the same reasons. The next development will be hybrid Diesel-electric trucks.

interesting things

/tmp /mnt/bind bind bind 0 0

Today I discovered that the above syntax works in the /etc/fstab file. This enables a bind mount of /tmp to /mnt/bind which effectively makes /mnt/bind a hard link to /tmp. The same result can be achieved by the following command, but last time I tried (quite some time ago) it didn't seem to work in /etc/fstab - but now it works in both SUSE and Debian.

mount --bind /tmp /mnt/bind

Also I recently discovered that 0.0.0.0 is an alias to 127.0.0.1. So for almost any command that takes an IP address you can use either address with equal results (apart from commands which interpret the string and consider 0.0.0.0 to be invalid). I can't think of any benefit to using this, and challenge the readers to post a comment (or make their own blog post if they so wish) demonstrating it's utility.

AMD developer Center

This morning I received an email from the AMD Developer Center advising me that I need to fill out their NDA if I want access to their development machines.

I have a vague recollection that when AMD64 was first released I was very keep to get access to such hardware and had applied to AMD for access to their machines.

Of course now the second-hand market is full of AMD64 machines and I've got one in my server room so it's not as useful as it once was. I don't even know why AMD would still run a developer center given that everyone who wants AMD64 machines can cheaply buy as many as they want and organizations such as Sourceforge and Debian provide access to such machines for their members.

While I appreciate what AMD is doing, it probably would be best if companies could adopt a standard timeout for electronic correspondence. If someone doesn't follow up for X months then you should assume that they are not interested.

comment-less blogs

Are comment-less blogs missing the spirit of blogging?

It seems to me that the most significant development about blogging is the idea that anyone can write. Prior to blogs news-papers were the only method of writing topical articles for a mass audience. To be able to write for a news-paper you had to be employed there or get a guest writing spot (not sure how you achieve this but examples are common).

Anyone can start a blog, if there is a community that you are part of which has a planet then it's not difficult to get your blog syndicated and have some reasonable readership. Even the most popular planets have less readers than most small papers, but that combined with the ease of forwarding articles gives a decent readership.

It seems to me that the major characteristic that separates a blog from an online newspaper is the low entry requirements, anyone can create one.

Every news-paper that is remotely worth reading has a letters column to publish feedback from readers. Of course it's heavily moderated and getting even 50% of your letters published is something to be proud of. But it does create a limited forum to discuss the articles that are published.

It seems to me that creating a blog and denying the readers the ability to comment on it is in some ways making the blog less open than a news-paper column. When such blogs are aggregated in a community planet feed it seems that they go against the community spirit. It also drives people to make one-line blog posts in response, which I regard as a bad thing.

The comments on my blog are generally of a high quality, I've had a few anonymous flame comments - but you have to learn to deal with a few flames if you are going to use the net, and people who are afraid to publish their real name to a flame don't deserve much attention. I've had one comment which might have been an attempt to advertise a product (so I deleted it just to be safe). But apart from that the comments are generally very good. I've learned quite a few useful things from blog comments, sometimes I mention having technical problems in blog posts and blog comments provide the solution. Other times they suggest topics for further writing.

There are facilities for moderated blog comments that some people use. If you have a really popular blog then it's probably a good idea to moderate the comments to avoid spam, but I'm not that popular yet and most people who blog will never be so popular. At this time blog moderation would be more trouble for me than it's worth.

In conclusion I believe that the web should be about interactive communication in all areas, it should provide a level playing field where the participation of all individuals is limited only by time and ability. Refusing comments on blogs is a small step away from that goal.

what defines a well operating planet?

At OSDC Mary Gardiner gave a talk titled The Planet Feed Reader: Better Living Through Gravity. During the course of the presentation she expressed the opinion that short dialog based blog entries are a sign of a well running planet.

Certainly if blog posts respond to each other then there is a community interaction, and if that is what you desire from a planet then it can be considered a good thing. Mary seemed focussed on planets for internal use rather than for people outside the community which makes the interaction more important.

However I believe that planets are not a direct substitute for mailing lists. On a mailing list you can reply to a message agreeing with it and expect that the same people who saw the original message will see your reply. Blogs however are each syndicated separately so a blog post in response to someone else's blog should be readable on it's own. A one line post saying "John is right" provides little value to people who don't know who John is, especially if you don't provide a link to John's post that you agree with.

On Planet Debian there have been a few contentious issues discussed where multiple people posted one-line blog entries. I believe that the effective way to communicate their opinions would either be to write a short essay (maybe 2-3 paragraphs) explaining their opinion and the reasons for it, or if they have no new insight to contribute then they should summarise the discussion.

I believe that a planet such as Planet Debian or Planet Linux Australia should not only be a forum for people who are in the community but also an introduction to the community for people who are outside. AOL posts don't help in this regard.

One final thing to note is that blogs already do have a feature for allowing "me too" responses, it's the blog comment facility...

PS Above is a picture of day 59 of the beard, it was taken on the 5th of December (I've been a little slack with beard pictures).

quantum evolution

On several occasions in discussions about life etc friends have mentioned the theory that quantum mechanics dictates the way our cells work. In the past I have not been convinced. However this site http://www.surrey.ac.uk/qe/Outline.htm has a very well written description of the theory which is very compelling.

some questions about disk encryption

On a mailing list some questions were asked about disk encryption, I decided to blog the answer for the benefit of others:

What type of encryption would be the strongest? the uncrackable if you will? im not interested in DES as this is a US govt recommendation - IDEA seems good but what kernel module implements this?

The US government (which incidentally employs some of the best cryptologists in the world) recommends encryption methods for data that is important to US interests (US military and banking operations for starters). Why wouldn't you want to follow those recommendations? Do you think that they are putting back-doors in their own systems?

If they were putting in back-doors do you think that they would use them (and potentially reveal their methods) for something as unimportant as your data?

I think that if the US military wanted to apply a serious effort to breaking the encryption on your data then you would have an assortment of other things to worry about, most of which would be more important to you than the integrity of your data.

I've read some good things about keeping a usb key for system boot so that anything on the computer itself is unreadable without the key - but thats simply just a physical object - I'd like both the system to ask for the passphrase for the key as well as needing the usb key

I believe that can be done with LUKS, however it seemed broken last time I experimented with it so I've stuck with the older operation of cryptsetup.

What kind of overheads does something like this entangle? - will my system crawl because of the constant IO load of the disk?

My laptop has a Pentium-M 1.7GHz and a typical laptop drive. The ratio of CPU power to hard drive speed is reasonable. For most operations I don't notice the overhead of encryption, the only problem is when performing CPU intensive IO operations (such as bzip compression of large files). When an application and the kernel both want to use a lot of CPU time then things can get slow.

More recent machines have a much higher ratio of CPU power to disk IO as CPU technology has been advancing much faster than disk technology. A high-end desktop system might have 2-3x the IO capacity
of my machine, but a single core would have 2-3x the computer power of the CPU in my laptop and for any system you might desire nowadays 2 cores is the minimum. Single-core machines are still on sale and still work well for many people - I am still deploying Pentium-3 machines in new installations, but for machines that make people drool it's all dual-core in laptops and one or two dual-core CPUs in desktop systems (with quad core CPUs on sale soon).

If you want to encrypt data on a P3 system with a RAID array (EG a P3 server) then you should expect some performance loss. But for a typical modern desktop system you shouldn't expect to notice any overhead.

IDE hard drives

I just lent two 80G IDE drives to a friend, and he re-paid me with 160G drives. Generally I don't mind people repaying hardware loans with better gear (much better than repaying with the same gear after a long delay and depreciation), but this concerns me.

My friend gave me the 160G drives because he can't purchase new 80G drives any more, his supplier has nothing smaller than 160G. I have some very reliable machines that I don't want to discard which won't support 160G drives - I'm not even sure that they would boot with them! Now I'm going to have to stock-pile 40G disks.

The machines I am most concerned about are my Cobalt machines. They are nice little servers that are quiet and use only 20W of electricity!

It's a pity that there aren't any cheap flash storage devices that connect to an IDE bus. If I could get my Cobalt machines running with flash storage they would be even more quiet and energy efficient while not being at risk of mechanical damage, and I doubt that flash storage will exceed 40G of capacity for a while.

Update: I've set a new personal record for rapid comments on a blog entry, all telling me that it is possible to get CF to IDE adapters. Thanks for the information, I appreciate it and will consider it for some machines. The problem however is that the price of a CF to IDE adapter plus the cost of a CF card of suitable size is moderately high (more than the cheaper hard drives), while CF capacity generally is only just usable for a mainstream Linux distribution.

These factors combine to make CF-IDE devices an option for only certain corner cases, not really an option to replace all the hard drives in machines that matter to me. I will probably use it for at least one of my Cobalt machines though.

Update2: Julien just informed me of the new Samsung flash-based laptop drives that will have capacities up to 16G (or 32G according to other web sites). I'm now trying to discover where to buy them.

The Squirrel and the Grasshopper

There's a story going around the neo-con blogs titled "The Squirrel and the Grasshopper". It was forwarded to me by a business associate with the claim that it's "right on the money". It's strange that someone could be considered to be "right on the money" for Australia when essentially the same text is posted in the UK, New Zealand, and Sweden (from a 30 second google search - I'm sure that the neo-cons in other countries have posted it too).

The following are the neo-con ideas promoted by the story in question:

1. To mis-represent a local main-stream political party that is known for representing workers (the ALP in the case of Australia) as being extremist and associated with Greenpeace (an organization that is out of favor at the moment and disliked by many people who vote for main-stream parties).
2. To spread the "liberal press" lie that wing-nuts like to believe. Any analysis of the press will show that most multi-national media organizations are quite biased towards the right-wing groups.
3. Making false claims about the legal system to drive support for recent fascistic legal changes. In the case of Australia this means allowing employers to lay off employees and immediately re-hire them at a lower rate, allowing employees to be laid off if factory equipment breaks down, and for almost any reason you can imagine. Driving the idea that the judges are incompetent and therefore imposing legislation to remove judicial discretion is an important step in removing civil rights.
4. Claims that the government is communist and takes the property from the middle-classes and gives it to unworthy people. In fact the opposite is true (for Australia at least). Large companies and wealthy individuals are routinely given community property. The toll roads are the best example of this, the government closes public roads that can be used as an alternative to a toll road, and then politicians get paid off after they leave office. Far from taking money from people who work (as the neo-con propaganda claims) the government allows big corporations to do so with impunity. The Australian government (as many governments in first-world countries) has been becoming increasingly fascistic recently.
5. The claim that asylum seekers are terrorists. If the government wanted to stop terrorism then they would cease involvement with those parts of the world. However only plebians (people like us) are likely to be hurt by terrorism so the government has little motivation to stop it - it's good for winning elections! By joining the invasion of Iraq the Australian government helped al Quaeda establish new training bases while also giving al Quaeda (and related organizations) a reason to target Australia. Also whenever a war is started people will be forced to leave their homes and seek asylum else-where. If you don't want asylum seekers seeking entry to your country then you don't want to mess up other countries and force people to flee.
6. Support for the "war on drugs". That war has been at best a stale-mate and generally a loss for a century now. The approach that is being adopted experimentally of legal supply of hard drugs to addicts seems to have more promise. Incidentally the breaches in border security that are established for the purpose of drug smuggling are available for any illegal purpose that pays enough - if al Quaeda wanted to smuggle weapons into a first-world country they would probably get drug dealers to do it for them. I'll blog more on this topic in future.

There you have it, The Squirrel and the Grasshopper covered all of the neo-con propaganda bases apart from the pro-Christian angle.

For the benefit of anyone who is thinking of forwarding on a "parable" in future, the first thing you might want to do is a google search on it. Search for comments and also search for who is promoting it. If a message you are considering forwarding is being promoted by people who are obviously racist or who discriminate against people on the basis of religion then you might consider whether you want to associate yourself with them by forwarding the message.

Debian SE Linux policy bug

checkmodule -m -o local.mod local.te
semodule_package -o local.pp -m local.mod
semodule -u local.pp

Save the following policy as local.te and then run the above commands to make semodule work correctly and to also allow restorecon to access the console on boot.

module local 1.0;

require {
        class chr_file { read write };
        class fd use;
        type restorecon_t;
        type tmpfs_t;
        type initrc_t;
        type semanage_t;
        role system_r;
};

allow restorecon_t tmpfs_t:chr_file { read write };
allow semanage_t initrc_t:fd use;

SE Linux on Debian in 5 minutes

Following from my 5 minute OSDC talk yesterday on 5 security improvements needed in Linux distributions I gave a 5 minute talk on installing SE Linux on Debian etch. To display the notes I formatted them such that they were in 24 line pages and used less at a virtual console to display them. The ultra-light laptop I was using has only 64M of RAM which isn't enough for a modern X environment and I couldn't be bothered getting something like Familiar going on it.

After base install you install the policy and the selinux-basics package:

# apt-get install selinux-basics selinux-policy-refpolicy-targeted
The following extra packages will be installed:
checkpolicy libsemanage1 mime-support policycoreutils python python-minimal
python-selinux python-semanage python-support python2.4 python2.4-minimal
selinux-utils
Suggested packages:
python-doc python-tk python-profiler python2.4-doc logcheck syslog-summary
The following NEW packages will be installed:
checkpolicy libsemanage1 mime-support policycoreutils python python-minimal
python-selinux python-semanage python-support python2.4 python2.4-minimal
selinux-basics selinux-policy-refpolicy-targeted selinux-utils
0 upgraded, 14 newly installed, 0 to remove and 0 not upgraded.
Need to get 6362kB of archives.
After unpacking 41.5MB of additional disk space will be used.
Do you want to continue [Y/n]?

The package install process also configures the policy for the machine. The next step is to label the filesystems, this took 26 seconds on my Celeron 500MHz laptop with 20,000 files on an old IDE disk. The time is in proportion to number of files, often bottlenecked on CPU. A more common install might have 5* as many files with a 5* faster CPU so 30 seconds is probably common for labelling. See the following:

# fixfiles relabel

Files in the /tmp directory may be labeled incorrectly, this command
can remove all files in /tmp.  If you choose to remove files from /tmp,
a reboot will be required after completion.

Do you wish to clean out the /tmp directory [N]? y
Cleaning out /tmp
/sbin/setfiles:  labeling files under /
matchpathcon_filespec_eval:  hash table stats: 14599 elements, 14245/65536 buckets used, longest chain length 2
/sbin/setfiles:  labeling files under /boot
matchpathcon_filespec_eval:  hash table stats: 19 elements, 19/65536 buckets used, longest chain length 1
/sbin/setfiles:  Done.

The next step is to edit /boot/grub/menu.list to enable SE Linux, auditing, and put it in enforcing mode:

title   Debian GNU/Linux, kernel 2.6.17-2-686
root    (hd0,1)
kernel  /vmlinuz-2.6.17-2-686 root=/dev/x selinux=1 audit=1 ro enforcing=1
initrd  /initrd.img-2.6.17-2-686

Then reboot.

After rebooting view the context of your shell, note that the login shell will have a domain of unconfined_t when the targeted policy is used:

# id -Z
system_u:system_r:unconfined_t

Now let's view all processes that are confined:

# ps axZ |grep -v unconfined_t|grep -v kernel_t|grep -v initrc_t
LABEL                             PID TTY   STAT   TIME COMMAND
system_u:system_r:init_t            1 ?     Ss     0:02 init [2]
system_u:system_r:udev_t         1999 ?     S.s    0:01 udevd --daemon
system_u:system_r:syslogd_t      3306 ?     Ss     0:00 /sbin/syslogd
system_u:system_r:klogd_t        3312 ?     Ss     0:00 /sbin/klogd -x
system_u:system_r:apmd_t         3372 ?     Ss     0:00 /usr/sbin/acpid -c /etc
system_u:system_r:gpm_t          3376 ?     Ss     0:00 /usr/sbin/gpm -m /dev/i
system_u:system_r:crond_t        3402 ?     Ss     0:00 /usr/sbin/cron
system_u:system_r:local_login_t  3423 tty1  Ss     0:00 /bin/login --
system_u:system_r:local_login_t  3424 tty2  Ss     0:00 /bin/login --
system_u:system_r:getty_t        3425 tty3  Ss+    0:00 /sbin/getty 38400 tty3
system_u:system_r:getty_t        3426 tty4  Ss+    0:00 /sbin/getty 38400 tty4
system_u:system_r:getty_t        3429 tty5  Ss+    0:00 /sbin/getty 38400 tty5
system_u:system_r:getty_t        3430 tty6  Ss+    0:00 /sbin/getty 38400 tty6
system_u:system_r:dhcpc_t        3672 ?     S.s    0:00 dhclient3 -pf /var/run/

The initial install of policy inserts modules to match installed software, if you install new software then you need to add new modules with the semodule command:

# semodule -i /usr/share/selinux/refpolicy-targeted/apache.pp
security:  3 users, 7 roles, 824 types, 67 bools
security:  58 classes, 11813 rules
audit(1165532434.664:21): policy loaded auid=4294967295
# semodule -i /usr/share/selinux/refpolicy-targeted/bind.pp
security:  3 users, 7 roles, 836 types, 68 bools
security:  58 classes, 12240 rules
audit(1165532467.874:22): policy loaded auid=4294967295

Note that the security and audit messages come from the kernel via printk, it is displayed on console login but you need to view the system log if logged in via ssh or running an xterm. Now you have to relabel the files that are related to the new policy:

# restorecon -R -v /etc /usr/sbin /var/run /var/log
restorecon reset /etc/bind context system_u:object_r:etc_t->system_u:object_r:named_zone_t
restorecon reset /etc/bind/named.conf context system_u:object_r:etc_t->system_u:object_r:named_conf_t
[...]
restorecon reset /etc/apache2 context system_u:object_r:etc_t->system_u:object_r:httpd_config_t
restorecon reset /etc/apache2/httpd.conf context system_u:object_r:etc_runtime_t->system_u:object_r:httpd_config_t
[...]
restorecon reset /usr/sbin/named context system_u:object_r:sbin_t->system_u:object_r:named_exec_t
restorecon reset /usr/sbin/apache2 context system_u:object_r:sbin_t->system_u:object_r:httpd_exec_t
restorecon reset /usr/sbin/rndc context system_u:object_r:sbin_t->system_u:object_r:ndc_exec_t
restorecon reset /usr/sbin/named-checkconf context system_u:object_r:sbin_t->system_u:object_r:named_checkconf_exec_t
[...]
restorecon reset /var/run/bind context system_u:object_r:var_run_t->system_u:object_r:named_var_run_t
restorecon reset /var/run/bind/run context system_u:object_r:var_run_t->system_u:object_r:named_var_run_t
restorecon reset /var/run/bind/run/named.pid context system_u:object_r:initrc_var_run_t->system_u:object_r:named_var_run_t
restorecon reset /var/run/motd context system_u:object_r:initrc_var_run_t->system_u:object_r:var_run_t
restorecon reset /var/run/apache2 context system_u:object_r:var_run_t->system_u:object_r:httpd_var_run_t
restorecon reset /var/run/apache2/cgisock.3558 context system_u:object_r:var_run_t->system_u:object_r:httpd_var_run_t
restorecon reset /var/run/apache2.pid context system_u:object_r:initrc_var_run_t->system_u:object_r:httpd_var_run_t
restorecon reset /var/log/apache2 context system_u:object_r:var_log_t->system_u:object_r:httpd_log_t
restorecon reset /var/log/apache2/error.log context system_u:object_r:var_log_t->system_u:object_r:httpd_log_t
restorecon reset /var/log/apache2/access.log context system_u:object_r:var_log_t->system_u:object_r:httpd_log_t

The -v option to restorecon causes it to give verbose output concerning it's operations. Often you won't do it in real use, but it's good to illustrate the use.

Now you have to restart the daemons:

# killall -9 apache2
# /etc/init.d/apache2 start
Starting web server (apache2)....
# /etc/init.d/bind9 restart
Stopping domain name service...: bind.
Starting domain name service...: bind.

Apache and BIND now run in confined domains, see the following ps output:

system_u:system_r:httpd_t   3833 ?     Ss     0:00 /usr/sbin/apache2 -k start
system_u:system_r:httpd_t   3834 ?     S      0:00 /usr/sbin/apache2 -k start
system_u:system_r:httpd_t   3839 ?     Sl     0:00 /usr/sbin/apache2 -k start
system_u:system_r:httpd_t   3841 ?     Sl     0:00 /usr/sbin/apache2 -k start
system_u:system_r:named_t   3917 ?     Ssl    0:00 /usr/sbin/named -u bind

It's not particularly difficult. I covered the actual install of SE Linux in about 1.5 minutes. I had considered just ending my talk there on a note of "it's so easy I don't need 5 minutes to talk about it" but decided that it was best to cover something that you need to do once it's installed.

If you want to know more about SE Linux then ask on the mailing list (see http://www.nsa.gov/selinux for subscription details), or ask on #selinux on freenode.

some advice for job seekers

A member of the free software community recently sent me their CV and asked for assistance in getting a job. Some of my suggestions are globally applicable so I'm blogging them.

Firstly I recommend that a job seeker doesn't publish their CV on the net in an obvious place. Often you want to give different versions to different people, and you don't necessarily want everyone to know about the work you do. I can't imagine any situation in which a potential employer might view a CV on the net if it's available but not ask for one if it isn't there. If you are intensively looking for work (IE you are currently between jobs) then I recommend having a copy of your CV in a hidden URL on your site. This means that if you happen to meet a potential employer you can give them a URL so that they can get your CV quickly, but the general public can't view it. A final problem with publishing your CV is that it may cause disputes with former colleagues (EG if you describe yourself as the most skilled programmer in the team then a former colleague who believes themself to be more skillful might disagree).

Next don't put your picture on your CV. In some jurisdictions it's apparently illegal for a hiring officer to consider your appearance. If there are many CVs put forward for the position then it may be easier to just discard yours because of this. There is absolutely no benefit to having the picture, unless of course you are applying for a job as an actor. Incidentally I've considered applying for work as an movie extra. The amount of effort involved is often minimal (EG pretend to drink beer in the back of a bar scene) and the pay is reasonable. It seems like a good thing to do when between computer contracts.

I write my CV in HTML and refuse to convert it. If a recruiting agent can't manage to use IE to print my CV then they are not competent enough to represent me. If a hiring manager can't manage to view my CV with IE then I don't want to report to them. However I recommend against using HTML features that make a document act in any way unlike a word-processor file. There should be no frames or CSS files so there is only one file to email, and the text should be all on one page so the PGDN and PGUP keys can scroll through all the content. Tables, bold, and italic are good, fonts are a minor risk. Colors are bad.

Recruiting agents will often demand that your CV be targeted for the position that you are applying for. I often had complaints such as "I see only sys-admin skills not programming". To solve this I wrote my CV in M4 and used a Makefile to generate multiple versions at the same time. If a recruiter wants a version of my CV emphasising C programming and using Linux then I've already got one ready!

These are just a few thoughts on the topic based on a CV that I just saw. I may write more articles about getting jobs in the computer industry if there is interest.

OSDC

Yesterday I gave a presentation at OSDC in Melbourne about my Postal mail server benchmark suite. The paper was about my new benchmark program BHM for testing the performance of mail relay systems and some of the things I learned by running it. I will put the paper on my Postal site in the near future and also I'll release a new version of Postal with the code in question very soon.

Today at OSDC I gave a 5 minute talk on 5 things that need to be improved in the security of Linux distributions.

1. The fact that unprivileged programs often inherit the controlling tty of privileged programs which permits them to use the TIOCSTI ioctl to insert characters in the keyboard buffer. I noted that with runuser and a subtle change to su things have been significantly improved in this regard in Fedora, but other distributions need work (and Fedora can go further in this regard).
2. A polyinstantiated /tmp should be an option that is easy to configure for a novice sys-admin. There have been too many attacks on data confidentiality and system integrity based on file name race conditions in /tmp, this needs to be fixed and must be fixable by novice sys-admins.
3. The capability system needs to be extended. 31 capabilities is not enough and the significant number of operations that are permitted by CAP_SYS_ADMIN leads to granting excessive privilege to programs.
4. The use of Xen on servers such that a domU is always used for applications should become common. Then if a compromise is suspected there will be better options for investigation.
5. SE Linux needs to be used more, particularly the strict policy and MCS. Use of the strict policy often reveals security flaws in other programs.

I'll blog about each of these in detail at some future time.

xen

I'm currently working on a little Debian Xen server, and I encountered a few problems that aren't documented.

The first problem I found was that serial ports don't work with a default Xen setup (as documented in a previous blog entry). However the solution to this turns out to be putting xencons=off on the kernel command-line for the dom0 kernel. This allows the dom0 kernel to see all the serial hardware. If I had wanted to use the serial ports from a domU then something else would need to be done, but as I have no need for this I didn't investigate the matter any further. Thanks to Brian May for discovering this for me.

Next in the early test relreses of etch the udev hotplug interface isn't enabled. So I had to add kernel.hotplug=/sbin/udevsend to the /etc/sysctl.conf file. A newer version of udev appears to fix this without modifications to the kernel.hotplug setting though. The symptom of this was the error "Error: Device 768 (vbd) could not be connected. Hotplug scripts not working" where 768 is the number for /dev/hda (the same message would occur with number 769 if you use /dev/hda1). I choose to use unpartitioned virtual disks such as /dev/hda and /dev/hdb in Xen because there is no benefit in partitions (my Xen instances are not doing enough to need more filesystems than there are virtual IDE disks) and because I don't desire the fake partition table thing that Xen apparently does.