From reading my web stats yesterday it seems that one Planet has polled by blog feed 1693 times over the first 14.25 days of this month. This is about 5 polls per hour. Another Planet has polled my blog 994 times for an average of about 3 hits per hour.
How frequently does it make sense to poll blogs?
Read the rest of this post on my new blog.
When I first put my new blog online I didn't think to set the custom permalinks option to avoid having /index.php in all URLs (which wastes a few bytes and looks nasty).
So I decided to change to better URLs but unfortunately many people have already bookmarked the bad URLs. I wanted to give a HTTP 301 redirection when someone uses the old index.php version (so that bookmarks get updated) and then redirect to the PHP file. Unfortunately having a redirection from ^/index.php to a version without it and then a local rewrite to include index.php again doesn't seem to work (any advice would be appreciated). So I put the following in my /etc/wordpress/htaccess file (the location for such things in Debian) so that foo.php is used instead where foo.php is a sym-link to index.php. I'm wondering whether I should file a bug report against the Debian package requesting that a sym-link be in the package to facilitate such things - if it's not possible to do what I desire without the symlink.
The rest of this post is on my new blog.
The URL http://linuxresource.lemonup.com/ currently has a mirror of my blog. Disregarding the DMCA take-down notice I sent them a week ago (which is also mirrored on their own site) they have again copied the content from my site without permission (I only allow non-commercial use). But this time they go even further and claim copyright over my text!
See the rest of this post on my new blog.
The war on comment-spam has now begun. It appears that Blogger might have some anti-spam measures of which I was unaware. Otherwise it’s a strange coincidence that I get a huge number of comment spams for extremely hard-core porn from the Ukraine so soon after starting a Wordpress blog.
About 24 hours before the spam attack there was a strange blog comment that linked to google (with no offensive or spammy content). It appears that leaving it online was my mistake, when I left that online for a day the spammer decided that I might also leave porn spam online. I arrived home this evening to find almost 100 spams in the form of comments and track-backs, and more arriving by the minute. So I used iptables to block a /20 related to the spam and things are quiet now.
The moral of the story is to delete anything unusual ASAP in case it encourages the idiots.
I’ve also tightened the anti-spam measures on my blog too.
I am starting to move my blog to my own Wordpress server. Here is the new URL for my main blog (feed), and here is the new URL for my Source-Dump blog (feed) which is now named just "dump".
Wordpress gives me the power to change all aspects of my blog's operation (including adding plug-ins). It also allows me to correctly display greater-than and less-than characters (the Perl script I use for converting them is at this post - it's short now but will probably grow).
Hopefully the new blog will also solve the date problems that some Planet readers have been complaining about.
I will briefly put the same content on both the old and new blogs, when I'm fully confident in the new blog I'll stop updating the old one and try to get all Planet installations changed. Anyone who wants to convert their Planet installation to my new blog now is welcome to do so.
I have just updated my previous post about licenses and also explicitely licensed my blog. Previously I had used a Creative-Commons share-alike license for lecture notes to allow commercial use and had not specified what the license is for my blog apart from it being free for feeds (you may add it to a planet without seeking permission first).
Unfortunately the operators of a site named lemonup.com decided to mirror many of my blog posts with Google AdWords. The site provides no benefit to users that I can discover and merely takes away AdWords revenue from my site. It has no listed method of contacting the site owner so it seems that blogging about this and letting them read it on their own site is the only way of doing so. :-#
I'm happy for Technorati to mirror my site as they provide significant benefits to users and to me personally. I am also happy for planet installations that include my blog among others to have a Google advert on the page (in which case it's a Google advert for the entire planet not for my blog post).
Also at this time I permit sites to mirror extracts of my articles. So for example the porn blogs that post paragraphs of my posts about topics such as "meeting people" with links to my posts don't bother me. I'm sure that someone who is searching for porn will not be happy to get links to posts about Debian release parties etc - but that's their QA issue not a license issue. I am aware that in some jurisdictions I can not prevent people from using extracts of my posts - but I permit this even in jurisdictions where such use is not mandated by law.
Lemonup: you may post short extracts (10% or one paragraph) of my posts with links to the original posts, or you may mirror my posts with no advertising at all. If those options are not of interest to you then please remove all content I wrote from your site.
If I enter "a < b" in blogger then it works, but if I want the < symbol to be next to some other text (EG for a #include line in C source) then it treats it as a HTML tag. The HTML code for a < symbol also doesn't work. This doesn't work regardless of whether I try entering HTML in the HTML editor or entering text in the "Compose" editor. I could deal with this problem if it forced me to strictly use one of the two editors available, but it fails in both!
Do other blog server programs have problems like this? I think that I need to change my blog server just to allow posting C source!
Also do any of the common blog servers allow a file upload? For most of my posts I do all the editing offline, so I'd rather just upload a HTML file instead of pasting the text into the blog editor (and then manually fixing the situations where it's idea of formatting differs from mine). If such an upload mode also supported getting a file via HTTP then that would be convenient too. The option of editing a file on a remote server with vi, exporting it via Apache, and then getting it to the blog server via HTTP would work well for me in some important situations.
Update: It sucks more than I thought. The feed misses the "less than" character between "a" and "b" in the above paragraph. My list of requirements in this regard now includes the ability to use such characters in a feed. Actually I want to go the whole hog and be able to include samples of HTML in my blog entries and have them display correctly in the feed.
I posted the NSA paper titled The Inevitability of Failure on my source-dump blog at this URL. If you want to get the background to SE Linux then this is the paper to read.
Submitting big documents (62K) to Blogger turns out to be surprisingly painful. It's a pity that Blogger doesn't support editing sections (as MediaWiki does) so every edit requires that the entire document be transferred back and forth. Do other blog servers make this easier?
It seems that the swik.net site is mirroring all my blog posts. The site seems to be doing some good things in terms of spreading information about free software and has a good presentation that makes such information easy to read. Also having a backup of my blog posts also could be handy if blogger ever does the wrong thing.
However it is a little annoying that when I write a blog post that refers to one of my older posts it will get a link back to swik.net. This is an annoyance for readers who want to see posts that link to mine from outside my blog. So I've been deleting those links when I notice them.
Also someone from Brazil has been linking to my posts, which is a good thing. Their blog also causes my blog to list theirs as a link which is also fine. However the problem is that their blog seems to detect me as being from an English speaking country and gives me an English version of the blog (rather than the presumably Portuguese version that has the link to my article). Assuming that someone speaks English because they reside in Australia is a bad idea, and breaking links is a worse one. So I've been deleting those links from my blog as they are of no use to people who are detected as English speakers (which comprises the vast majority of my blog readers). When someone blogs about one of my posts I want to see what they wrote, even if all that I can read are the parts that are quoted from me!
Finally I've been deleting some comments containing URLs. It seems that there are quite a few people trying to advertise their businesses by posting comments that bear some vague relation to a blog post with their company's URL included. You have to try harder than that if you want to promote yourself on my blog.
From a suggestion on my previous blog entry I decided to test out google reader.
The first problem was that it caused Konqueror to SEGV in etch, I filed a bug report and switched to Firefox.
Next to add my feeds I had to either export them in OPML format or add them one at a time, there is no support for pasting in a list of URLs. If I was writing a RSS syndication program I would also make it parse the config files of some of the common programs, parsing a Planet config file is pretty easy.
I added a feed for a friend who's server seems to be down. While doing so I tried to add another feed, the google reader accepted the command to add the second feed but didn't actually do so - it was fortunate that I was pasting it in not typing it...
The killer issue is that it seems to be impossible to merge feeds. I want to read both Planet Linux Australia and Planet Debian, there are some people who are on both planets (EG me). So it makes no sense to do anything other than display both of them in the same view.
At this time it seems that google reader is unsuitable for my use. However it is a fairly slick system and I imagine that it would work quite well for people who have different needs to me. If you want to read the blogs of a few friends then it probably works really well. It just seems not to work well for a set of meshed communities (Debian developers and Linux users in Australia for example).
Please let me know if I somehow missed some configuration options to make google reader do what I want.
I just noticed that /usr/bin/planetplanet is using about 120M of RAM. This isn't currently a problem as I'm running it on a machine with 256M of RAM, however I would like to run my web server on a 96M Xen instance. 120M for planetplanet is probably going to cause bad performance on a web server with 96M of physical RAM allocated.
This is a serious problem for me as the Xen server in question can't be upgraded any more (the motherboard has as much RAM as it can handle).
Are there any other free syndication programs that use less memory?
Are there any good free syndication services that I could use instead of running my own planet?
Are comment-less blogs missing the spirit of blogging?
It seems to me that the most significant development about blogging is the idea that anyone can write. Prior to blogs news-papers were the only method of writing topical articles for a mass audience. To be able to write for a news-paper you had to be employed there or get a guest writing spot (not sure how you achieve this but examples are common).
Anyone can start a blog, if there is a community that you are part of which has a planet then it's not difficult to get your blog syndicated and have some reasonable readership. Even the most popular planets have less readers than most small papers, but that combined with the ease of forwarding articles gives a decent readership.
It seems to me that the major characteristic that separates a blog from an online newspaper is the low entry requirements, anyone can create one.
Every news-paper that is remotely worth reading has a letters column to publish feedback from readers. Of course it's heavily moderated and getting even 50% of your letters published is something to be proud of. But it does create a limited forum to discuss the articles that are published.
It seems to me that creating a blog and denying the readers the ability to comment on it is in some ways making the blog less open than a news-paper column. When such blogs are aggregated in a community planet feed it seems that they go against the community spirit. It also drives people to make one-line blog posts in response, which I regard as a bad thing.
The comments on my blog are generally of a high quality, I've had a few anonymous flame comments - but you have to learn to deal with a few flames if you are going to use the net, and people who are afraid to publish their real name to a flame don't deserve much attention. I've had one comment which might have been an attempt to advertise a product (so I deleted it just to be safe). But apart from that the comments are generally very good. I've learned quite a few useful things from blog comments, sometimes I mention having technical problems in blog posts and blog comments provide the solution. Other times they suggest topics for further writing.
There are facilities for moderated blog comments that some people use. If you have a really popular blog then it's probably a good idea to moderate the comments to avoid spam, but I'm not that popular yet and most people who blog will never be so popular. At this time blog moderation would be more trouble for me than it's worth.
In conclusion I believe that the web should be about interactive communication in all areas, it should provide a level playing field where the participation of all individuals is limited only by time and ability. Refusing comments on blogs is a small step away from that goal.
At OSDC Mary Gardiner gave a talk titled The Planet Feed Reader: Better Living Through Gravity. During the course of the presentation she expressed the opinion that short dialog based blog entries are a sign of a well running planet.
Certainly if blog posts respond to each other then there is a community interaction, and if that is what you desire from a planet then it can be considered a good thing. Mary seemed focussed on planets for internal use rather than for people outside the community which makes the interaction more important.
However I believe that planets are not a direct substitute for mailing lists. On a mailing list you can reply to a message agreeing with it and expect that the same people who saw the original message will see your reply. Blogs however are each syndicated separately so a blog post in response to someone else's blog should be readable on it's own. A one line post saying "John is right" provides little value to people who don't know who John is, especially if you don't provide a link to John's post that you agree with.
On Planet Debian there have been a few contentious issues discussed where multiple people posted one-line blog entries. I believe that the effective way to communicate their opinions would either be to write a short essay (maybe 2-3 paragraphs) explaining their opinion and the reasons for it, or if they have no new insight to contribute then they should summarise the discussion.
I believe that a planet such as Planet Debian or Planet Linux Australia should not only be a forum for people who are in the community but also an introduction to the community for people who are outside. AOL posts don't help in this regard.
One final thing to note is that blogs already do have a feature for allowing "me too" responses, it's the blog comment facility...
PS Above is a picture of day 59 of the beard, it was taken on the 5th of December (I've been a little slack with beard pictures).
I notice that Planet Linux Australia has been changed to not list the feeds URLs, instead it displays the HTML pages for the blogs.
I believe this is a bad idea as some people want to get a list of feeds for the blogs that are aggregated without having to visit all the blog sites and do it manually. One of the many reasons for doing this is for a blog server that has intermittent net access, it might be down at the moment which prevents me from adding it to my feed list. Another reason is that some people (such as me) want to automatically get a list of all feeds from the planet to add to their own personal planet configuration.
I am blogging this not to criticise the administrators of Planet Linux Australia or even to inform them (I have already send them an email). My point is to prevent other people from doing the same thing. At this time I am not sure whether this change in Planet Linux Australia was deliberate, a result of a bug in Planet, or a mistake in configuration (maybe a default changed unexpectedly).
Another planet related surprise that I received today was to notice that my blog appears to have been removed from Planet Fedora. I'm not sure why this happened, one possibility is that removing my blog was regarded as the solution to the problem of it displaying incorrectly (the better solution being to upgrade the Planet software as was done on Planet Debian). Another possibility is that my post about Gratis vs Libre was regarded as criteria for removal. If my blog was removed from Planet Debian or Planet Linux Australia then I would be able to ask the administrators about this (they have email address links conveniently located). Planet Fedora has no such link, so I guess I'll have to wait for a blog comment to find out.
I've included a day 19 beard picture, I was planning to do one yesterday but a design meeting for a VOIP project ran late and I ran out of time. I'll write a post about VOIP in the near future.
Are blogs and conference speeches inherently about self promotion? If so is that a bad thing?
Recently I mentioned my Planet configuration on a mailing list where most people don't track new technology. Some people viewed blog entries for the first time as a result of this and then claimed that blogs appeared to be mostly about self-promotion.
It seems to me that people offer conference speeches to promote technology that they believe in (which often equates to self-promotion as such people are well known to be associated with the technology in question), for promoting themself or their own business, or to promote the company that employes them. Of these categories of talk the worst ones are those which are given to promote a company, it's especially bad when a talk starts with "the guy who was supposed to give this talk was called to a client so I'm doing it instead" - this indicates how much the company cares for the quality of the talk. Someone who is promoting themself will care about doing a reasonable job. Someone who promotes their favourite technology will usually give a great talk! But corporations rarely get the idea that a good quality talk which makes little mention of their products is the most effective advertisement. Of recent times Google seems to be the best example of a company which gets this idea, at many conferences there are Google employees giving talks about various technologies not directly related to Google operations without any direct sales pitch. Everyone who attends such talks gets the message - Google has hired many smart people and has them working on cool things.
It seems that blogs are often written with similar motivations to conference presentations but with no control over the topic and less quality control. The difference of course is that a blog doesn't get a forum the way a talk which is accepted by a conference will. So a corporate blog has to be really good to get readers.
PS I've added the day15 beard picture to this entry, it was taken on the 22nd of October, but I had only just got around to GIMPing it and uploading it.
I believe that adding blogging technology to bug tracking systems (such as the Debian BTS and the Red Hat Bugzilla) offers significant benefits for users and developers.
It seems that there is only one significant difference between the features offered by blog software and the features required by bug trackers, blog entries are owned by the person making the post (the owner of the blog) and BTS entries are owned by the package owner. This is a minor technical difference, adding the ability for anonymous users (or users who have authenticated via an email address or other method) to create new entries in a blog owned by someone else is a minor feature change. Once the entry is created only comments would be accepted, and comments could be made in the usual manner with moderation etc.
The benefits of using blogging software (or extending current BTS systems) would be to have RSS feeds of bug reports and comments on bugs, include a single feed of all bug reports (IE blog entries) and updates (IE comments on blog entries) for a single blog (IE package) - this functionality doesn't seem to be common in blog software but surely will be soon.
This would allow Planets of recent bug reports in distributions or in areas. EG a SE Linux planet installation could syndicate the bug feeds for packages in Fedora and Debian as well as other sources of SE Linux information. The SE Linux bug feeds could include all bugs with the selinux tag in all blogs on the BTS, thus a single Planet configuration could easily cover all new development and debugging.
Another configuration possibility would be to have a single blog for all bugs in a distribution and to have tags for each package, this would become messy though as tags have to be used for issues that are not specific to packages (EG a "selinux" tag would list bugs in all packages that relate to SE Linux).
Tags could be changed by any user (possibly with some restrictions), and there could be special tags for release-critical bugs or for bugs in different stages of QA (for example the Red Hat Enterprise Linux approval process could be managed by adding certain restricted tags to the bug - this would also allow getting a feed of all bugs that have passed certain approval milestones). This would make it easy to get a list of all bugs in a certain approval category.
Current BTS systems such as the Debian BTS and the Red Hat Bugzilla could be extended to syndicate content. As they already have web interfaces and support automatically sending updates by email it should be easy to add the basic level of such support to them. But it might give a better result to go the other way and modify a multi-user blog system such as Wordpress-MU to have extra features for bug management.
BTS systems typically have bug ID numbers that increase sequentially while blog software typically gives something like http://blog.example.com/YYYY/MM/blog-entry-name.html when it's often more convenient to have something like http://bug.example.com/number. Of course it would not be difficult to have a little database that allows creating sequential numbers to map to arbitrary URLs in a similar manner to tinyurl.com. This could be either integrated into blog software (so that it says "bug #1234 has been created" or be available on demand (click a link to get a sequential number assigned). Incidentally using characters and digits for the short name of the bug (as done by tinyurl.com) would allow four characters to represent 1679616 bugs, this is enough for all bugs in the Debian and Red Hat bug databases combined. With tinyurl type technology there is nothing preventing us from having a single system creating unique IDs for all bugs in all distributions of Linux with a four character index (which is easier to remember than the 6 sigit numbers used in Debian and Red Hat at the moment).
I realise that most blog software will store the entries in a database which has sequential numbers assigned. So doing a database lookup to convert a human-readable URL into a sequential index when we have just done another database lookup to convert a sequential index into a human readable form is inefficient, but that's the way of things, computers do things inefficiently so that humans can work in ways that are efficient for them.
Note that throughout this post I refer to the Planet aggregation system, but really any of the aggregation systems can be used. One of the benefits of this is that people can use mailing lists, Google's reader, Planet, or any other reader they wish with the feeds. RSS feeds can be read by many programs and the user gets to choose which works best for them.
I've read quite a few blog posts about someone leaving Debian and whether they should remain on Debian planet. An official policy on these matters has now been posted which stated what I expected, if you feel that you belong and meet technical criteria then you are welcome.
Not that this solves much, the next debate will be about what content is suitable for Debian-planet with the expected answer being "anything which meets technical criteria and doesn't offend many people or break any laws". I've already had some comments on my blog from people who want me to change topics. I don't know if other people get this or whether doing an average of one post per day gets me more attention from the loons.
I started blogging after leaving Red Hat. I considered asking for my blog to be added to the Fedora Planet, but wasn't sure whether I would be posting much about it. I just checked and it seems that my old Advogato blog is aggregared on the Fedora Planet and there is no mailto URL on that site to allow me to get it changed. I've just put a final blog entry on Advogato to inform everyone of the change.
I'm not sure if it's worth adding my blog to the Fedora syndication. I have just decided to change my main desktop machine from Rawhide to Debian/unstable. The reason is that Fedora is mostly a Gratis distribution and Debian is more Libre. For most computer users there is no real difference as they don't have the skills to use the liberty that Debian offers. But for people who can code (note that we are in a small minority of computer users) the difference is significant.
The final issue that forced me to this decision is this bugzilla entry about Xen. In Debian there are kernels for Xen on i686, Xen on AMD K7, and Xen on i686 with vserver (doesn't Xen make vserver redundant?). In Fedora there will be one Xen kernel which won't boot on the machine that is most important to me and which ironically is the machine that was issued to me by Red Hat (and sold to me when I left).
This issue of a lack of choice is quite understandable from the Red Hat Enterprise Linux side of things. It's OK to say to a customer who wants to pay for a RHEL-AS license that they need a machine less than 3 years old if they want to use all the features. Adding new kernels adds support costs and I think that most RHEL customers want to have a smaller set of supported options with a higher level of support. I often recommend RHEL to clients and I will recommend that clients use Xen on RHEL-5 - and that they purchase recent hardware to do so.
But for home and hobby use it's a different matter. I provide all the support I need, I can compile my own kernels without much effort - but it saves time to have someone else do it. Fedora simply lacks choice here by design. I still support a bunch of Fedora and RHEL machines and will still develop RPMs for them. I will put everything I develop under http://www.coker.com.au/rpms/ for anyone who wants it.
Given that if the Fedora Planet people want to syndicate my blog I am more than happy to have them do so. I don't dislike Fedora, in fact I still recommend that people use it. It's just that Debian suits my personal needs better than Fedora does. I expect that I'll have more Debian content than Fedora content on my blog, but there will also be a lot of Linux content that's not distribution specific.
It will be interesting to see what the Fedora Planet people do.
For fun I just set up my own Planet. Reading both Planet Debian and Planet Linux Australia is a bit of a drag due to the overlap. So I wrote a little Perl script to extract the feeds from both those sources and generate a Planet configuration. My planet is publicly available in case anyone is interested. Also you will notice that I have a formatting problem on my planet, if anyone has some advice on planet templates (I'm using the Debian package) then please let me know.
While playing with Planet I noticed that my blog has one of the largest file sizes of all the blogs from Planet Debian and Planet Linux Australia. That would be partly due to writing blog entries of moderate size and trying to maintain an average of one post per day. But also I imagine that it woul d be partly due to the blogger configuration.
I changed my main blog page from 14 days to 7 days (which took the index.html size from 80K down to 40K). But strangely I can't seem to change the number of days that are to be kept in the RSS feed file which remains at about 80K.
It seems to me that the feed mechanism is badly designed in this regard. A more efficient mechanism would be to send an XML file that describes the other blog entries, which could then be downloaded IFF they meet the criteria that are desired by the syndication software (and IFF they have not already been downloaded). If the web server and the syndication program are correctly configured then the requests could be chained on the same TCP connection for no loss of performance under any situation when compared to the way things currently work.
Also as many (most?) RSS and ATOM feed files are generated from a database it might be difficult to preserve the creation time, and thus I expect that most web caching is also broken. I haven't verified this though.
Also it would be handy if there was a mechanism for syndication programs to request notification of changes and for a blog server to push content to a syndication server. I have Planet running every 6 hours, some of the blogs I read are updated once per week, apparently my Planet server does 28 downloads of the entire XML file for every change. This might not sound so bad, but there are planets which run every hour and would therefore have 168 downloads per week.
Please let me know if you have any ideas of how to alleviate these problems, or if there are already solutions of which I am not aware.
There is currently a big debate in progress in Debian. I am not going to mention any specifics because too much of it has already been blogged (maybe in the same syndication in which you read my blog).
I think that the way things are going is more an illustration of the failings of mailing list culture than of failings of Debian. Maybe another mechanism would be more productive in leading towards a solution.
One option that occurred to me is debate via wiki. If each side had a wiki page that they could modify then in a small amount of time we should get a set of two main consensus opinions which would each be explained clearly and summarised well. Then with two options clearly expressed the people who have less strong opinions could decide which option they favor. For this to be a quick solution honorable behaviour would be required from all people involved, if people start trying to sabotage the other group's wiki entries then it would significantly increase the time taken to achieve things.
Another possibility that occurred to me is debate via blog. The quality of blog postings is expected to be a lot higher than that of mailing list discussions as all posts are tied to the author's public image. Writing content-free messages on a mailing list is easy, but every blog entry needs to stand on it's own to a certain extent and anyone who writes flames in most of their blog entries will probably find that the readers like it less than the readers of a typical mailing list.
Maybe when an issue is recognised as highly contentious a few people could blog about it and then form groups to develop wikis to promote their views. A debate might start out with five or more different competing views, some of them would merge until there were only two main opinions being pushed. Then once the two remaining groups had sorted out their positions a vote would be easier to arrange.
What do you think?
Previously I asked for advice about running an Intranet blog, and running an Internet blog with hosting for friends.
In response to the question about running a small Intranet blog the recommendations were strongly for Wordpress, with a mention of Ikiwiki as well. One of the features that I consider desirable is for software to be reasonably popular which means that support is often easier to obtain. So Wordpress is my main candidate at this time for Intranet use. I'll install Wordpress and probably won't try anything else unless Wordpress fails in some significant way (which seems unlikely).
In response to my question about a blog server for serious blogging again Wordpress was well recommended. There is also a version of Wordpress in beta called Wordpress MU that supports blog server operations such as wordpress.com. Although I didn't mention it before I have had some ideas of starting my own server along such lines so Wordpress again does well.
Over the next few weeks I will start playing with Wordpress and Wordpress MU. If things go well I'll move my blog away from blogspot and to a domain I own in the near future.
