The LANGUAGE environment variable can accept multiple fallback languages (at least if your commands are using gettext), so if your main LANG is, say, es, but you also speak fr, then you can use LANGUAGE=es:fr.

But what happens when you main LANG is en, so for example your LANGUAGE looks like en:es:de? You'll notice some message that used to be in perfect English before using the multi-language fallback now seem to be shown randomly in es or de.

Well, it is not random. The thing is, since English tends to be the de-facto language for the original strings in a program, it looks like almost nobody provides an en translation, so when fallback is active, almost no programs will show messages in English.

For example, this is my Debian testing system with roughly 3.5K packages installed:

$ dpkg -l |wc -l
3522
$ ls /usr/share/locale/en/LC_MESSAGES/ | wc -l
12

Only 12 packages have a plain English locale. en_GB does a bit better:

$ ls /usr/share/locale/en_GB/LC_MESSAGES/ | wc -l
732

732 packages. This is still lower than both en and de:

$ ls /usr/share/locale/es/LC_MESSAGES/ | wc -l
821
$ ls /usr/share/locale/de/LC_MESSAGES/ | wc -l
820

The weird thing is packages as basic as psmisc (providing, for example, killall) and coreutils (providing, for example, ls) don't have an en locale, and psmisc doesn't provide es. This is why at some point it seemed like a random locale was being used. I had something like LANGUAGE=en_GB:en_US:en:es:de and I use KDE as my desktop environment. KDE seems to be correctly translated to en_GB, so I was seeing most of my desktop in English as expected, but when using killall, I got errors in German, and when using ls, I got errors in Spanish.

If you don't provide other fallback languages, gettext will automatically fall back to the C locale, which is the original strings embedded in the source code, which are usually in English, and this is why if you don't provide fallback languages (other than English at least), all will work in English as expected. Of course if you use C in your fallback languages, before any non-English language, then they will be ignored as the C locale should always be present, so that's not an option.

I find it very curious that this issue has almost zero visibility. At least my searches for the issue didn't throw any useful results. I had to figure it all out by myself like in the good old pre-stackoverflow times...

I usually don't do reviews for anything, but I want to write a few points about this phone, in part for folks out there to know, but also as some sort of internal reminder of the things I've been finding.

I should say before anything else that I'm basically comparing this phone against my previous one, a Samsung Galaxy S4 (I9505) using CyanogenMod.

The Elephone P900 is a super tempting device. Here are the main reasons why I chosen this phone (in bold my hard requirements, in italics things I didn't really care about but it was a good opportunity to try out):

• It has 4GB of RAM and fast CPU (octacore)
• It supports memory expansion (micro SD up to 256GB 8-)
• It has a very high screen to body ratio (about 83%, for comparison the Nexus 5 has 71% and the iPhone 5s has 61%). I was looking for a 5 inch phone, so to go for a 5.5 inch one, the overall size of the phone had to be as small as possible.
• It has USB Type-C (I thought if I'm getting a new phone, better to have the new shiny no-non-sense connector)
• It comes with Android 6 (I want either that or to be supported by CM)
• Decent battery (3000mAh, which I expected to last for a full day of intensive use)
• It has at least one "navigation button" (I don't want to lose part of my screen with software buttons)
• It has a Sony camera sensor with f/2.0 and laser focus, which is supposed to be of decent quality really fast to do focus. That said, I read some reviews not speaking well about the camera
• It has a rouged back (my S4 was a bit slippery)
• It costs less than €250 (here in Germany)
• It has wireless charging and quick charging
• It has a fingerprint reader
• It has a quite high-res frontal camera (8 megapixel)
• It supports dual-SIM although the second SIM shares the slot with the memory expansion, so is not something I'll be able to use anyway

So, after using it for about a couple of days, these are my findings:

The good:

• The build quality is very nice, it really looks like a high end phone. More than my old S4 (which is made of plastic, while the P9000 is metallic).
• Is very light, even when it's a few grams extra compared to the S4, you can't really feel it. For offering 0.5 extra inches of screen is quite impressive.
• It is fast. I wasn't expecting to notice a difference with the S4 really, basically because I don't feel the S4 is slow. But you can tell the difference. The P9000 is snappier.
• It looks beautiful (at least for my minimalistic taste). I never care much about looks, but I really like this phone (much more than the S4).
• Despite the big screen and feeling a bit too big at first, the size seems manageable and the extra screen space is useful.
• Now having a fingerprint reader will probably be a requirement for my next phone. I encrypt my phone and use a longish password to unlock it. Being able to unlock it securely with just one touch is a huge gain.
• It can be rooted. It took me a while to find the right flasher for Linux (you need the latest version of it), but I could do it, and even TWRP is available for it already, which gives me some hope about better ROMs, and maybe even CM, appearing in the future.

The Bad:

• The fingerprint reader sucks. I've seen a video review complaining about it, and even for this guy complaining it worked much better than for me. I would say in my case it succeeds reading my fingerprint about less than 20% of the time. I even registered my fingerprint like five 5 times, using different finger positions and it still fails most of the time, and after 3 or 5 failures you have to wait 30 seconds before being able to retry.
• The screen is not bright enough for a sunny day. You can still see the screen, but it's not as bright as the one in my old S4.
• The camera pretty much sucks too. The f/2.0 I don't know where is it, pictures are always quite noisy. The auto-focus is not faster than my old S4. The sensor is supposed to be good, so I guess they just screwed it with the lenses. Or maybe is a firmware thing? But I doubt it.
• The sound really SUCKS. I never thought about it before. Even when I listen to music a lot, I never had a good year and never could pick up on bad quality recordings for example. Is a blessing. But with this phone I noticed. It sounds like crap (and I'm not talking about the speaker, which is understandable, I'm talking about plugging earphones). When I noticed I thought it might be the album. I tried another one, and another one, and finally I compared the same files in my old S4 and... Oh boy. This new phone's audio just SUCKS SO BAD. It's the phone. I would say this was the final deal breaker for me.
• The battery can't take an intense full day of usage. It's basically the same as my old S4 (and I want an improvement on this area). If I use it just for a few messages and most of the time inside with WiFi, it can last 2 full days. If I take it outside using mobile data, and listen to music during a hold day, it barely last for a day. If I add to that using maps and the GPS having the screen on more time, like when I'm traveling, it can barely last more than half a day.
• The Android version is missing some features that I thought it was pure Android (not CM), like the Ambient Display (shows notifications in a dimmed screen), the LiveDisplay (adjust the screen color temperature according to the time of the day) and the Do Not Disturb mode(s). The keyboard doesn't support swiping (major drawback for me).
• The touchscreen is not very sensitive. I can tell the difference with the S4. Maybe the one in the S4 is too sensitive, sometimes it reacts without even contacting the glass, but in the P9000 sometimes I feel I have to press the glass too much to get it reacting. Some gestures are harder to do because of this (like swipe-up to unlock).
• Only one navigation button. Even when is better than nothing, I found much more convenient having 3 navigation buttons like the S4 provides.
• No multicolor notification light. The navigation button on the bottom also serves as a notification light, but it has only one color and the frequency can't be configured either (AFAIK). The S4 has a multi-color led, which let you know what kind of notification is there before you even look at the screen.
• USB Type-C is not popular enough yet. Even when this is not the phone's fault, I realized we are not there yet. Micro USB cables are everywhere out there. You'll never miss one. With Type-C you better carry your cable everywhere or buy a bunch, because you are all alone now.

So, even when the external quality is amazing and, even when I never cared about looks, it looks extremely nice too, it looks like the low price tag has to come from somewhere, and that somewhere is the internal components, which seems like they are not the best.

Still the quality-price ratio is quite impressive IMHO, on paper you have the same specs as an iPhone 6, or Samsung S7, at less than half the price. But I think I prefer to spend an extra few bucks to get higher internal components qualities (specially with the sound), so I will probably return this phone and continue looking for one. Also I miss my CM too much. I think I will have to settle for an older phone that's is supported by CM.

TODO:

• Fingerprint unlock and screen gestures makes the phone never enter deep sleep, nice features but until it's fixed it might be better to disable them, at least when you know you'll need some juice.
• Fingerprint starts working better after some use (only 2 or 3 attempts are needed)
• WiFi consumes more battery than mobile data (WTF!?)
• RoDrIgUeZsTyLe™ MODPACK V1.1
  • Sound is saved (probably is not amazing, but I don't notice the obvious creepiness anymore). Praise the Lord!
  • Touchscreen is more sensitive, not as the S4 but definitely an appreciable improvement.
  • Battery life improved a lot. One day of moderate activity (for me) and still about 65% battery left. I used the phone for 12 hours after fully charged, and my usage pattern was: about 6 hours outside (using mobile data), about 40 minutes of music playing + GPS working in high accuracy mode. The rest inside using WiFi. I also did some messaging and internet use, but nothing too intensive. BetterBatteryStats reports: 66% (~8h) deep sleep (which is still low, I wonder what this phone could last if it were more aggressive about going to deep sleep), 8% (~1h) screen on. Wifi running 100% of the time (~12h). Battery consumption average was 3%/h. My guess is that with a similar usage, the S4 I would probably ended the day with not more than 30% or 40% battery.
• Ways to improve the battery life and fix other stuff: Xposed framework, but not supported yet. For example with "Amplify Battery Extender" I could disable the wakelock for the fingerprint or NlpWakelock. Wakelock Terminator might also help (also needs Xposed)
• For now using DisableService to disable NlpService from MTK NLP Service, the location seems to keep working fine. Another option is to put the GPS in device-only mode to avoid the NPL service from running.
• Install Xposed using Eragon 2.0 ROM:
  1. Install Xposed Material installer (3.0 alpha4) http://forum.xda-developers.com/xposed/material-design-xposed-installer-t3137758
  2. Edit /etc/init.d/07permissive and comment out the sleep 60
  3. Install Xposed framework v82 (not a newer one otherwise settings will force close)
• Battery life saved via update from 2016-05-31. Fingerprint reader and WiFi don't keep the phone awake anymore (5~10% awake when screen is off with both enabled).

This is mostly an article I want to save for myself about simplicity. It was originally written by Mark Ramm in the context of a Python web framework I used (TurboGears). The original article seems to be gone, but you can still find it in the Archive.org's Wayback Machine.

Here is a transcription:

What is Simplicity?

(May 31st, 2006 by Mark Ramm)

Simplicity is knowing when one more rock would be too many, and one less rock would be too few. But it’s not just knowing the right number of rocks, it’s also knowing which rocks are right, and how to arrange them.

As Brad reminds us, simplicity is not achieved merely by making something easier, or less complex.

Take away all the complexity, all the difficulty, and all of the details from anything and what you are left with is not simple: it’s just boring.

On the other hand, Simplicity embraces exactly the right details, the right difficulties, the right complexity, but because everything is tied together in the right way, you are left with a sense of clarity, and a sense that everything belongs exactly where it is. Simplicity is achieved when everything means something.

In other words, simplicity is defined by what you add — clarity, purpose, and intentionality — not by what you remove.

For those of us who write software, simplicity is not a simple thing to learn. Writing the TurboGears book and working with the amazing group of people who contribute to the project has been a learning experience for me. Everybody is focused on making the web development simpler — and it’s amazing how much experience and depth of understanding is necessary to create a simple interface. It’s easy to build an interface that solves 80% of the problem, or an interface that solves 200% of the problem, but it is hard to solve just the right problem, and to do it in a clean, clear, way.

Of course, every project has warts, and TurboGears re-uses other projects which also have warts. So there’s no way I can say that TurboGears has arrived. But the will is there, and the journey sure has been productive for me.

Maybe you heard about the new album from The Black Keys. Maybe you didn't. In any case, I don't want to talk about the album (which is good BTW), I want to talk about the album cover:

See how bad it looks? Now click on the image and see how good it looks (in terms of quality, the album cover is pretty ugly anyway :P). The thing is, this stupid pattern is very hard to compress, so even using a JPG quality of 90%, you get a quite big file size and a pretty crappy image quality (126KB for a 500x500 image is quite a lot, 294KB for PNG using compression 9). If you look at the big image, even the colors are different, so the image makes resizing algorithms also go nuts, the image looks darker (or is this just an ilusion because of the changed relationship between both colors?).

Try it yourself, download the image, resize it, save it with different formats and qualities.

Coincidence? I guess not.

On Anniversary of Aaron Swartz's Tragic Passing, Leading Internet Groups and Online Platforms Announce Day of Activism Against NSA Surveillance.

Participants including Access, Demand Progress, the Electronic Frontier Foundation, Fight for the Future, Free Press, BoingBoing, Reddit, Mozilla, ThoughtWorks, and more to come, will join potentially millions of Internet users to pressure lawmakers to end mass surveillance -- of both Americans and the citizens of the whole world.

https://thedaywefightback.org/

9 months ago I decided to try Flattr. I created an account, put some money on it, started flattring and made myself flattrable. But nothing happened. Also sometimes you don't know if the people you are flattring will even reclaim your flattrs (in services that automatically provides flattr links).

Conclusion, I got quite disappointed. But today I see the light again, as I received my first and only flattr (for eventxx). Thanks whoever you are, anonymous hero, you brought hope again to humanity :P

Anyway, I'll try to give it a shot again, and try to keep the wheel moving.

You should do that too.

Fucking awesome, be patient for the first minute...

I learned that Flattr, a social micropayment service that I've been overlooking for a long time, was created by some of the founders of The Pirate Bay after watching TPB AFK.

I'm trying to donate (or pay) more and more to people using alternative means to produce stuff, like artists using CC licenses or software developers working with free licenses (I already bought a copy of the movie :). I feel like I have to get more involved to keep the wheel spinning and help people keep doing stuff, cutting the intermediaries as much as possible.

I don't know why I had some resistance to get into Flattr, maybe is because Facebook made me hate anything that have a thumbs up, or a +1 or counter, but knowing the history behind it a little better encouraged me to finally get an account and start using Flattr. And is really nice. Is much easier than going through Paypal each time a want to give some bucks to someone, and allows you to even make very small donations.

I recommend to see this introductory video:

I also decided to flattr-ize all my website, each project individually and even this blog. Not exactly for economical reasons (I think very few people know about anything I do so I don't really expect to earn any money from this), but as another way to spread the word. Also, I'm really curious about what I just said, I really wonder if there is someone out there grateful enough to make even a micro-donation to anything I do or did :)

Anyway, I would like to recommend to do the same, if you do something great, add a Flattr button to what you do, and if you like something out there and it has a Flattr, click it. Let's see if it helps to keep the wheel spinning :)

Fuck! I bought this extremely nice ultrabook, the Toshiba Satellite Z830-10J, about an year ago, and I've been experiencing some problems with CPU frequency scaling.

At one point I looked and looked for kernel bugs without much success. I went through several kernel updates in the hope of this being fixed, but never happened.

It seemed that the problem wasn't so bad after all, because I only got the CPU frequency locked down to the minimum when using the ondemand scaling governor, but the conservative was working apparently OK.

Just a little more latency I thought, is not that bad.

Recently I received an update on a related bug and I thought about giving it another shot. This mentioned something about booting with processor.ignore_ppc=1 to ignore some BIOS warning about temperature to overcome some faulty BIOS, so I thought on trying that.

But before doing, if this were a real BIOS problem, I thought about looking for some BIOS update. And there was one. The European Toshiba website offered only a Windows program to do the update though, but fortunately I found in a forum a suggestion about using the non-European BIOS upgrade instead, which was provided also as an ISO image. The problem is I don't have a CD-ROM, but that shouldn't stop me, I still have USB sticks and hard-drives, how hard could it be? I failed with UNetbootin but quickly found a nice article explaining how to boot an ISO image directly with grub.

BIOS upgraded, problem not fixed. So I was a about to try the kernel parameter when I remembered I saw some other article when googling desperately for answers suggesting changing some BIOS options to fix a similar problem.

So I though about messing with the BIOS first instead. The first option I saw that looked a little suspicious was in:

PowerManagement
   -> BIOS Power Management
      -> Battery Save Mode (using custom settings)
         -> Processor Speed
            <Low>

That is supposed to be only for non-ACPI capable OS, so I thought it shouldn't be a problem, but I tried with <High> instead.

WOW!!!

I start noticing the notebook booting much faster, but I thought maybe it was all in my mind...

But no, then my session opened way faster too, and everything was extremely faster. I think maybe about twice as fast. Everything feels a lot more responsive too. I can't believe I spend almost an year with this performance penalty. FUCKING FAULTY BIOS. I didn't make any battery life comparisons yet, but my guess is everything will go well, because it should still consume very little power when idle.

Anyway, lesson learned:

Less blaming to the kernel, more blaming to the hardware manufacturers.

But I still want to clarify that I love this notebook. I found it a perfect combination between features, weight and battery life, and now that it runs twice as fast (at least in my brain), is even better.

Hope this is useful for someone.

I live in Germany, and unfortunately there is something in Germany called GEMA, which basically censor any content that "might have copyrighted music".

Among the sites being censored are Grooveshark (completely banned) and YouTube (only banned if the video might have copyrighted music according to some algorithm made by Google). Basically this is because GEMA want these company to pay royalties for each time some copyrighted song get streamed). AFAIK Grooveshark don't want to pay at all, and Google want to pay a fixed fee (which is what it does in every other country), because it makes no sense to do otherwise, since anyone can just endlessly get a song streamed over and over again just to be paid.

Even when the model is debatable, there is a key aspect and why I call this censorship: not all the banned content is copyrighted or protected by GEMA.

• In Grooveshark there are plenty of bands that release their music using more friendly license, like CC.
• There are official videos posted in YouTube by the bands themselves and embedded in the band official website that is banned by GEMA.
• There are videos in YouTube that doesn't have copyrighted music at all, but they have to cover their asses and ban everything suspicious just in case.
• The personal videos that do have copyrighted music get banned completely, not only muted.

These are just the examples that pop on my mind now.

There are plenty of ways to bypass this censorship and they are the only way to access legal content in Germany that gets unfairly banned, not only harming the consumers, but also the artists, because most of the time having their music exposed in YouTube only spreads the word and do more good than harm.

HideMyAss is a popular choice for a web proxy. But I like SecurityKiss, a free VPN (up to 300MB per day), because it gives a more comprehensive solution.

But here comes the technical chalenge! :) I don't want to route all my traffic through the VPN, or to have to turn the VPN on and off again each time I want to see the censored content. What I want is to see some websites through the VPN. A challenge that proved to be harder than I initially thought and why I'm posting it here.

So the final setup I got working is:

• OpenVPN to use SecurityKiss free VPN service.
• Dante Socks Server to route any application capable of using a socks server through the VPN.
• FoxyProxy Firefox extension to selectively route YouTube and Grooveshark through the Socks proxy.

And here is how I did it (in Ubuntu 12.10):

sudo apt-get install openvpn

client
dev tunsk
proto udp

# VPN server IP : PORT
# (pick the server you want from the README file in the bundle)
remote 178.238.142.243 123
nobind

ca   /etc/ssl/private/openvpn-sk/ca.crt
cert /etc/ssl/private/openvpn-sk/client.crt
key  /etc/ssl/private/openvpn-sk/client.key

comp-lzo yes
persist-key
persist-tun

user openvpn
group nogroup

auth-nocache
script-security 2

route-noexec
route-up "/etc/openvpn/sk.setup.sh up"

down "/usr/bin/sudo /etc/openvpn/sk.setup.sh down"

mknod /dev/net/tunsk c 10 200

echo 'AUTOSTART="sk"' >> /etc/default/openvpn

adduser --system --home /etc/openvpn openvpn

#!/bin/sh
# Based on:
# http://serverfault.com/questions/345111/iptables-target-to-route-packet-to-specific-interface

#exec >> /tmp/log
#exec 2>> /tmp/log.err
#set -x

# Config
uid=skvpn
gid=skvpn
mark=100
table=$mark
priv_dev=br-priv

env_file="/var/run/openvpn.sk.env"
umark_rule="OUTPUT -t mangle -m owner --uid-owner $uid -j MARK --set-mark $mark"
gmark_rule="OUTPUT -t mangle -m owner --gid-owner $gid -j MARK --set-mark $mark"
masq_rule="POSTROUTING -t nat -o $dev -j SNAT --to-source $ifconfig_local"

up()
{
        # Save environment
        env > $env_file

        # Route all traffic marked with $mark through route table $table
        ip rule add fwmark $mark table $table

        # Make all traffic go through the VPN gateway in route table $table
        ip route add table $table default via $route_vpn_gateway dev $dev

        # Except for the internal traffic
        ip route | grep "dev $priv_dev" | \
                        xargs -n1 -d'\n' echo ip route add table $table | sh

        # Flush route tables cache
        ip route flush cache

        # Mark packets originated by processes with owner $uid/$gid with $mark
        iptables -A $umark_rule
        iptables -A $gmark_rule

        # Prevent the packets sent over $dev getting the LAN addr as source IP
        iptables -A $masq_rule

        # Relax the reverse path source validation
        sysctl -w "net.ipv4.conf.$dev.rp_filter=2"
}

down()
{
        # Restore and remove environment
        . $env_file
        rm $env_file

        # Since the device is already removed, there is no need to clean
        # anything that was referencing the device because it was already
        # removed by the kernel.

        # Delete iptable rules
        iptables -D $umark_rule
        iptables -D $gmark_rule

        # Delete route table and rules for lookup
        ip rule del fwmark $mark table $table

        # Flush route tables cache
        ip route flush cache
}

if test "$1" = "up"
then
        up
elif test "$1" = "down"
then
        down
else
        echo "Usage: $0 (up|down)" >&2
        exit 1
fi

sudo adduser --system --home /etc/openvpn --group skvpn
sudo visudo

openvpn ALL=(ALL:ALL) NOPASSWD: /etc/openvpn/sk.setup.sh

sudo service openvpn start

sudo -u skvpn wget -qO- http://www.securitykiss.com | grep YOUR.IP

cd /tmp
for suffix in .orig.tar.gz -3.dsc -3.debian.tar.bz2
do
   wget http://ftp.de.debian.org/debian/pool/main/d/dante/dante_1.2.2+dfsg$suffix
done
sudo apt-get build-dep dante-server
dpkg-source -x dante_1.2.2+dfsg-3.dsc
cd dante_1.2.2+dfsg
dpkg-buildpackage -rfakeroot
cd ..
dpkg -i /tmp/dante-server_1.2.2+dfsg-3_amd64.deb

logoutput: syslog

internal: 10.1.1.1 port = 1080
external: tunsk

clientmethod: none
method: none

user.privileged: skvpn
user.unprivileged: skvpn
user.libwrap: skvpn

client pass {
        from: 10.1.1.0/24 port 1-65535 to: 0.0.0.0/0
        log: error # connect disconnect
}
client pass {
        from: 10.1.1.0/24 port 1-65535 to: 0.0.0.0/0
}

#generic pass statement - bind/outgoing traffic
pass {
        from: 0.0.0.0/0 to: 0.0.0.0/0
        command: bind connect udpassociate
        log: error # connect disconnect iooperation
}
#generic pass statement for incoming connections/packets
pass {
        from: 0.0.0.0/0 to: 0.0.0.0/0
        command: bindreply udpreply
        log: error # connect disconnect iooperation
}

sudo service danted start

^https?://(.*\.)?youtube\.com/.*$
^https?://(.*\.)?grooveshark\.com/.*$

Just a quick note because it took me ages to find out how to do it.

I don't really use the feature of pulseaudio that gives every application their own volume instead of manipulating the master volume directly, and lately it became more and more a problem, as I want to use applications like mpd or xbmc that allow remote controlling, and for that having separate volumes makes no sense.

I managed to fix it in mpd once, by using a mixed setup, using pulse as output, but the hardware alsa mixer, but for xbmc I couldn't find a way.

So that made me think if I really wanted the split volumes thingy, and the answer was no. After looking for hours for how to do it, the answer is pretty trivial. Just edit /etc/pulse/daemon.conf and change the flat-volumes option to yes.

You are welcome.

A little old, but always fun!

fffff.at

Reminds me a little of The Yes Men.

OK, first of all, this is pretty old (more than one year) but I just bumped into it and it seems interesting enough for me to post it.

Belle & Sebastian's singer, Stuart Murdoch, have posted his mind about Spotify and music streaming services, which apparently are ripping off artists way worse even than traditional record companies (see graph below).

I'll transcribe the part of the post found most interesting (well, it's actually almost the whole post) for convenience (bold added by me), but you can read the whole post for unbiased and complete information :)

[...] Ok, now my point, and probably my only important point: I’m certainly not against ‘the generation who no longer pay for music.’ That horse has bolted. And hey, I like that horse! It’s free and young and happy and doing its horsey thing.

What has had me conflicted is Spotify itself. Overnight, this thing appeared called Spotify, claiming it was a great idea, innovative, the saviour of the industry. From what I can gather, and no one has been able to tell me differently, it’s financed by a gathering of the top (ie. richest) people, from the top (ie. richest) record labels.

Overnight, the whole Belle And Sebastian back catalogue became available to stream, for anybody, for free, for good. We weren’t asked about it.

“How were you not asked?” I can imagine you would say. That’s exactly what I asked the record label. Their answer was not that informative. They mumbled something about a distribution company, that was under some umbrella; that it wasn’t up to them.

Can I just stress that Rough Trade is certainly not one of the aforementioned ‘richest’ record companies. I feel a bit bad for them. I’m gathering that they thought they had nothing to lose with the Spotify thing, that they had to try something new. (Kids, if there’s a less viable career choice than ‘independent recording artist’ at the minute, I would certainly say it was ‘independent record label.’)

Anyway, that’s enough of the angst. I’ve said it to the rest of the band, and I’ll say it again, “just because we’re in a band, it doesn’t make it a bloody pension plan”. We’ve had, and continue to have, a brilliant time making music and playing music and dreaming, and just about getting away with it. If it just got harder, then that’s because it should be hard. I think in the end it will make the music, the art, better.

I’m not even so much against Spotify. If they can get their model right, ie pay the bands something approaching appropriate amounts, then it will be all ok. I’m ready to throw my lot in with them; I mean, I use it now. And if I was 19 I would have used it too. (Would have used it to decide which vinyl/music to buy/see, as I’m sure lots of people still do)

It just seemed rich of them that they decided to charge everyone. They lured everyone in with ‘Our’ music (the royal ‘Our’), which they didn’t pay for, and now, probably because a shareholder somewhere is sitting in a Porsche, crying for a dividend, they’re going to charge money in our name. And I will eat my beloved black hat if we ever see a share. [...]

Another very interesting bit of information is the one provided in one of the comments, a nice graph about how much money artists get according to the distribution method, which I will also put here for convenience (I hope the author don't mind).

It would be nice to see an update including more open-license friendly services like Bandcamp, Magnatunes or Jamendo.

This is in the subway, there are huge banners in the bigger train stations too.

Esto está en el subte, también hay carteles enormes en las estaciones más grandes de tren.

Big ideals for small screens / Grandes ideales para pequeñas pantallas

[ N900 1/20s f/2.8 iso214 5.2 mm Auto ]

We work for you, not for shareholders / Trabajamos para usted, no para accionistas

[ N900 1/29s f/2.8 iso200 5.2 mm Auto ]

I don't like to trust my important data to big companies like Google. That's why even when I have a GMail, I don't use it as my main account. I'm also a little old fashion for some things, and I like to use Mutt to check my e-mail.

But GMail have a very useful feature, at least it became very useful since I moved to a country which language I don't understand very well yet, that's not available in Mutt: translation.

But that's the good thing about free software and console programs, they are usually easy to hack to get whatever you're missing, so that's what I did.

The immediate solution in my mind was: download some program that uses Google Translate to translate stuff, and pipe messages through it using a macro. Simple, right? No. At least I couldn't find any script to do the translation, because Google Translate API is now paid.

So I tried to look for alternatives, first for some translation program that worked locally, but at least in Ubuntu's repositories I couldn't find anything. Then for online services alternatives, but nothing particularly useful either. So I finally found a guy that, doing some Firebuging, found how to use the free Google translate service. Using that example, I put together a 100 SLOC nice general Python script that you can use to translate stuff, piping them through it. Here is a trivial demonstration of the script (gt, short for Google Translate... Brilliant!):

$ echo hola mundo | gt
hello world
$ echo hallo Welt | gt --to fr
Bonjour tout le monde

And here is the output of gt --help to get a better impression on the script's capabilities:

usage: gt [-h] [--from LANG] [--to LANG] [--input-file FILE]
          [--output-file FILE] [--input-encoding ENC] [--output-encoding ENC]

Translate text using Google Translate.

optional arguments:
  -h, --help            show this help message and exit
  --from LANG, -f LANG  Translate from LANG language (e.g. en, de, es,
                        default: auto)
  --to LANG, -t LANG    Translate to LANG language (e.g. en, de, es, default:
                        en)
  --input-file FILE, -i FILE
                        Get text to translate from FILE instead of stdin
  --output-file FILE, -o FILE
                        Output translated text to FILE instead of stdout
  --input-encoding ENC, -I ENC
                        Use ENC caracter encoding to read the input (default:
                        get from locale)
  --output-encoding ENC, -O ENC
                        Use ENC caracter encoding to write the output
                        (default: get from locale)

You can download the script here, but be warned, I only tested it with Python 3.2. It's almost certain that it won't work with Python < 3.0, and there is a chance it won't work with Python 3.1 either. Please report success or failure, and patches to make it work with older Python versions are always welcome.

Ideally you shouldn't abuse Google's service through this script, if you need to translate massive texts every 50ms just pay for the service. For me it doesn't make any sense to do so, because I'm not using the service differently, when I didn't have the script I just copy&pasted the text to translate to the web. Another drawback of using the script is I couldn't find any way to make it work using HTTPS, so you shouldn't translate sensitive data (you shouldn't do so using the web either, because AFAIK it travels as plain text too).

Anyway, the final step was just to connect Mutt with the script. The solution I found is not ideal, but works most of the time. Just add these macros to your muttrc:

macro index,pager <Esc>t "v/plain\n|gt|less\n" "Translate the first plain text part to English"
macro attach <Esc>t "|gt|less\n" "Translate to English"

Now using Esc t in the index or pager view, you'll see the first plain text part of the message translated from an auto-detected language to English in the default encoding. In the attachments view, Esc t will pipe the current part instead. One thing I don't know how to do (or if it's even possible) is to get the encoding of the part being piped to let gt know. For now I have to make the pipe manually for parts that are not in UTF-8 to call gt with the right encoding options. The results are piped through less for convenience. Of course you can write your own macros to translate to another language other than English or use a different default encoding. For example, to translate to Spanish using ISO-8859-1 encoding, just replace the macro with this one:

macro index,pager <Esc>t "v/plain\n|gt -tes -Iiso-8859-1|less\n" "Translate the first plain text part to Spanish"

Well, that's it! I hope is as useful to you as is being to me ;-)

usage: gt [-h] [--from LANG] [--to LANG] [--input-file FILE]
          [--output-file FILE] [--input-encoding ENC] [--output-encoding ENC]

Translate text using Google Translate.

optional arguments:
  -h, --help            show this help message and exit
  --from LANG, -f LANG  Translate from LANG language (e.g. en, de, es,
                        default: auto)
  --to LANG, -t LANG    Translate to LANG language (e.g. en, de, es, default:
                        en)
  --input-file FILE, -i FILE
                        Get text to translate from FILE instead of stdin
  --output-file FILE, -o FILE
                        Output translated text to FILE instead of stdout
  --input-encoding ENC, -I ENC
                        Use ENC caracter encoding to read the input, can be a
                        comma separated list of encodings to try, LOCALE being
                        a special value for the user's locale-specified
                        preferred encoding (default: LOCALE,utf-8,iso-8859-15)
  --output-encoding ENC, -O ENC
                        Use ENC caracter encoding to write the output
                        (default: LOCALE)

macro index,pager <Esc>t "v/plain\n|gt -IMS-GREEK,IBM-1148,UTF-16BE|less\n"

Some time ago CouchSurfing announced that they will become a socially responsible B-Corporation. In case you forgot about it, here is what the creators of this utopic project said then:

I believed them, and when everybody went bananas about it I thought there was some extremism and overreaction. After all, you need money to maintain the structure for a service like CS and if this change would help, it was fine with me as long as the spirit of the community was the same.

Unfortunately, now I think all these people were right. A few days ago CS announced a change in the terms of usage and privacy policy. The new ones include terms as stupid and abusive as:

5.3 Member Content License. If you post Member Content to our Services, you hereby grant us a perpetual, worldwide, irrevocable, non-exclusive, royalty-free and fully sublicensable license to use, reproduce, display, perform, adapt, modify, create derivative works from, distribute, have distributed and promote such Member Content in any form, in all media now known or hereinafter created and for any purpose, including without limitation the right to use your name, likeness, voice or identity.

They also removed all mention about being a socially responsible B-Corp, so, where is this heading at? I don't have a Facebook account because I appreciate my privacy and feel like FB never cared about it (among other things), but this terms of CS makes FB looks like the EFF!

Here is one of the many related discussions about the issue inside CS forums:

http://www.couchsurfing.org/group_read.html?gid=7621&post=13000298

People are even making complaints in their's respective country data protection and privacy agencies. But I see little sense in it, at least from a point of view of being part of CS. Even if they fix now the terms to be more reasonable, I don't want to be part of it any more.

So, from tomorrow all the content you leave in CS will be theirs forever, irrevocably. Unfortunately there is no way to opt out, or keep the old content under the old terms, so they give me no option but to remove all the content I don't want to give them perpetual, irrevocable, sublicensable, etc. rights to. And that's what I'm doing right now.

I think for now I will keep my account open (with fake information about me, even when I'm violation the terms and conditions which says I should provide truthful information about me) because there is still a great community behind it and I don't want to loose all contact with it. But my plan is to start using BeWelcome.org instead, hoping they don't eventually follow the same path CS did (their website is open source so at least there is a chance to clone the service if they do).

So, thanks for all the good times CS, and RIP!

Está en el edificio donde está la Embajada Argentina en Berlín.

Finally 1.0 is here, and in Extras-devel! The only important change since last release is a bug fix that prevented display blanking inhibition from properly work in devices configured with a display blanking timeout of less than 30 seconds (thanks cobalt1 for the bug report).

For more information and screenshots, you can visit the website.

You can download this release (binary package and sources) from here:

• https://llucax.com.nyud.net/proj/sadba/files/1.0/
• http://maemo.org/packages/view/status-area-displayblanking-applet/

But now you just might want to simply install it using the application manager.

You can also get the source from the git repository:

https://git.llucax.com/w/software/sadba.git

Please feel free to leave your comments and suggestions here or in the Maemo Talk Thread..

Final beta release for the Status Area Display Blanking Applet. Changes since last release:

• Show a status icon when display blanking is inhibited.
• Improve package description and add icon for the Application Manager.
• Add a extended description for display blanking modes.
• Update translation files.
• Code cleanup.

Also now the applet have a small home page and upload to Extras is on the way!

This is how this new version looks like:

You can download this 0.9 beta release (binary package and sources) from here: https://llucax.com.nyud.net/proj/sadba/files/0.9/

You can also get the source from the git repository: https://git.llucax.com/w/software/sadba.git

Please feel free to leave your comments and suggestions here or in the Maemo Talk Thread..

New pre-release for the Status Area Display Blanking Applet. New timed inhibition button that inhibit display blanking for an user-defined amount of time. Also there's been some code cleanup since last release.

You can download this 0.5 pre-release (binary package and sources) from here: https://llucax.com.nyud.net/proj/sadba/files/0.5/

You can also get the source from the git repository: https://git.llucax.com/w/software/sadba.git

Please feel free to leave your comments and suggestions here or in the Maemo Talk Thread..

New pre-release of my first Maemo application: The Status Area Display Blanking Applet. Now you inhibit display blanking without changing the display blanking mode. The GUI is a little rough compared with the previous version but it works. :)

You can download this 0.4 pre-release (binary package and sources) from here: https://llucax.com.nyud.net/proj/sadba/files/0.4/

You can also get the source from the git repository: https://git.llucax.com/w/software/sadba.git

Please feel free to leave your comments and suggestions here or in the Maemo Talk Thread..

New pre-release of my first Maemo application: The Status Area Display Blanking Applet. Now you can pick whatever blanking mode you want instead of blindly cycling through all available modes, as it was in the previous version.

You can download this 0.3 pre-release (binary package and sources) from here: https://llucax.com.nyud.net/proj/sadba/files/0.3/

You can also get the source from the git repository: https://git.llucax.com/w/software/sadba.git

Please feel free to leave your comments and suggestions here or in the Maemo Talk Thread..

Second pre-release of my first Maemo application: The Status Area Display Blanking Applet. No big changes since the last release just code cleanup and a bugfix or new features (depends on how you see it). Now the applet monitors changes on the current configuration, so if you change the display blanking mode from settings (or by any other means), it will be updated in the applet too.

You can download this 0.2 pre-release (binary package and sources) from here: https://llucax.com.nyud.net/proj/sadba/files/0.2/

You can also get the source from the git repository: https://git.llucax.com/w/software/sadba.git

Please feel free to leave your comments and suggestions here or in the Maemo Talk Thread..

Hi, I just wanted to announce the pre-release of my first Maemo "application". The Status Area Display Blanking Applet let you easily change the display blanking mode right from the status menu, without having to go through the settings.

This is specially useful if you have a short blanking time when you use applications that you want to look at for a long time without interacting with the phone and don't inhibit display blanking by themselves (for example a web browser, image viewer or some GPS applications).

You can download this 0.1 pre-release (binary package and sources) from here: https://llucax.com.nyud.net/proj/sadba/files/0.1/

You can also get the source from the git repository: https://git.llucax.com/w/software/sadba.git

Here are some screenshots (the application is highlighted so you can spot it more easily :) ):

Please feel free to leave your comments and suggestions.

I'll upload the package to extras-devel when I have some time to learn the procedure.

So, Peter Sundes from The Pirate Bay has been convicted to 1 year prison and 11 million euro. He lost the appeal too, so now he is looking for a last resort, a plea for pardon, a procedure where you can get a judicial sentencing undone by the political administration in exceptional circumstances.

The plea for pardon is not serious in the sense that he is not really doing so, he is denouncing an extremely corrupt and absurd trial. You can read the plea and find out, is long but really interesting how the trial makes no sense (besides what's your stand on file sharing, copyright, etc.).

If you believe the trial was unfair, you can sign this petition, it will probably be completely ignored, but hey, it only takes 2 seconds, worth trying.

This is just a quick fix for yesterday's release. Now mutt-nntp depends on mutt >= 1.5.21-5. This should allow having mutt-nntp installed with the standard distribution mutt package for both Debian and Ubuntu (please report any problems).

If you have Ubuntu 12.04 (Precise) and amd64 or i386 arch, just download and install the provided packages.

For other setups, here are the quick (copy&paste) instructions:

ver=1.5.21
deb_ver=$ver-5nntp3
url=https://llucax.com.nyud.net/proj/mutt-nntp-debian/files/latest
wget $url/mutt_$deb_ver.dsc $url/mutt_$deb_ver.diff.gz \
   http://ftp.de.debian.org/debian/pool/main/m/mutt/mutt_$ver.orig.tar.gz
sudo apt-get build-dep mutt
dpkg-source -x mutt_$deb_ver.dsc
cd mutt-$ver
dpkg-buildpackage -rfakeroot
# install any missing packages reported by dpkg-buildpackage and try again
cd ..
sudo dpkg -i mutt-nntp_${deb_ver}_*.deb

Now you can enjoy reading your favourite newsgroups and your favourite mailing lists via Gmane with Mutt without leaving the beauty of your packaging system. No need to thank me, I'm glad to be helpful (but if you want to make a donation, just let me know ;).

See the project page for more details.

A new version of Mutt with NNTP support is available. This version only moves Mutt with NNTP support to a separate package in the hopes of having a smoother interaction with the distribution packages (avoiding automatic updates with less hassle). Now a new mutt-nntp package is generated.

If you have Ubuntu 12.04 (Precise) and amd64 or i386 arch, just download and install the provided packages.

For other setups, here are the quick (copy&paste) instructions:

ver=1.5.21
deb_ver=$ver-5nntp2
url=https://llucax.com.nyud.net/proj/mutt-nntp-debian/files/latest
wget $url/mutt_$deb_ver.dsc $url/mutt_$deb_ver.diff.gz \
   http://ftp.de.debian.org/debian/pool/main/m/mutt/mutt_$ver.orig.tar.gz
sudo apt-get build-dep mutt
dpkg-source -x mutt_$deb_ver.dsc
cd mutt-$ver
dpkg-buildpackage -rfakeroot
# install any missing packages reported by dpkg-buildpackage and try again
cd ..
sudo dpkg -i mutt-nntp_${deb_ver}_*.deb

Now you can enjoy reading your favourite newsgroups and your favourite mailing lists via Gmane with Mutt without leaving the beauty of your packaging system. No need to thank me, I'm glad to be helpful (but if you want to make a donation, just let me know ;).

See the project page for more details.

Since there is not a lot of information on how to hack Maemo's address book to find some contacts with a mobile phone number, I'll share my findings.

Since setting up an environment to cross-compile for ARM is a big hassle, I decided to write this small test program in Python, (ab)using the wonderful ctypes module to avoid compiling at all.

Here is a very small script to use the (sadly proprietary) OSSO Addressbook library:

# This function get all the names in the address book with mobile phone numbers
# and print them. The code is Python but is as similar as C as possible.
def get_all_mobiles():

    osso_ctx = osso_initialize("test_abook", "0.1", FALSE)
    osso_abook_init(argc, argv, hash(osso_ctx))

    roster = osso_abook_aggregator_get_default(NULL)
    osso_abook_waitable_run(roster, g_main_context_default(), NULL)
    contacts = osso_abook_aggregator_list_master_contacts(roster)

    for contact in glist(contacts):
        name = osso_abook_contact_get_display_name(contact)
        # Somehow hackish way to get the EVC_TEL attributes
        field = e_contact_field_id("mobile-phone")
        attrs = e_contact_get_attributes(contact, field)
        mobiles = []
        for attr in glist(attrs):
            types = e_vcard_attribute_get_param(attr, "TYPE")
            for t in glist(types):
                type = ctypes.c_char_p(t).value
                # Remove this condition to get all phone numbers
                # (not just mobile phones)
                if type == "CELL":
                    mobiles.append(e_vcard_attribute_get_value(attr))
        if mobiles:
            print name, mobiles


# Python

import sys
import ctypes
# be sure to import gtk before calling osso_abook_init()
import gtk
import osso

osso_initialize = osso.Context

# Dynamic libraries bindings
glib = ctypes.CDLL('libglib-2.0.so.0')
g_main_context_default = glib.g_main_context_default
def glist(addr):
    class _GList(ctypes.Structure):
        _fields_ = [('data', ctypes.c_void_p),
                    ('next', ctypes.c_void_p)]
    l = addr
    while l:
        l = _GList.from_address(l)
        yield l.data
        l = l.next

osso_abook = ctypes.CDLL('libosso-abook-1.0.so.0')
osso_abook_init = osso_abook.osso_abook_init
osso_abook_aggregator_get_default = osso_abook.osso_abook_aggregator_get_default
osso_abook_waitable_run = osso_abook.osso_abook_waitable_run
osso_abook_aggregator_list_master_contacts = osso_abook.osso_abook_aggregator_list_master_contacts
osso_abook_contact_get_display_name = osso_abook.osso_abook_contact_get_display_name
osso_abook_contact_get_display_name.restype = ctypes.c_char_p

ebook = ctypes.CDLL('libebook-1.2.so.5')
e_contact_field_id = ebook.e_contact_field_id
e_contact_get_attributes = ebook.e_contact_get_attributes
e_vcard_attribute_get_value = ebook.e_vcard_attribute_get_value
e_vcard_attribute_get_value.restype = ctypes.c_char_p
e_vcard_attribute_get_param = ebook.e_vcard_attribute_get_param

# argc/argv adaption
argv_type = ctypes.c_char_p * len(sys.argv)
argv = ctypes.byref(argv_type(*sys.argv))
argc = ctypes.byref(ctypes.c_int(len(sys.argv)))

# C-ish aliases
NULL = None
FALSE = False

# Run the test
get_all_mobiles()

Here are some useful links I used as reference:

• http://wiki.maemo.org/PyMaemo/Accessing_APIs_without_Python_bindings
• http://wiki.maemo.org/Documentation/Maemo_5_Developer_Guide/Using_Generic_Platform_Components/Using_Address_Book_API
• http://maemo.org/api_refs/5.0/5.0-final/libebook/
• http://maemo.org/api_refs/5.0/5.0-final/libosso-abook/
• https://garage.maemo.org/plugins/scmsvn/viewcvs.php/releases/evolution-data-server/1.4.2.1-20091104/addressbook/libebook-dbus/?root=eds
• http://www.developer.nokia.com/Community/Discussion/showthread.php?200914-How-to-print-all-contact-names-in-N900-addressbook-Please-Help

Nice post by Paulo Coelho promoting the piracy of his own books via The Pirate Bay (whom reciprocally returns the favor).

The Pirate Bay starts today a new and interesting system to promote arts.

Do you have a band? Are you an aspiring movie producer? A comedian? A cartoon artist?

They will replace the front page logo with a link to your work.

As soon as I learned about it, I decided to participate. Several of my books are there, and as I said in a previous post, My thoughts on SOPA, the physical sales of my books are growing since my readers post them in P2P sites.

Welcome to download my books for free and, if you enjoy them, buy a hard copy – the way we have to tell to the industry that greed leads to nowhere.

Love

The Pirate Coelho

Go, search, download, read and if you like them, buy or show your appreciation in another way.

OK, after I recovered my phone without needing to reflash once, I was even much closer to do it again because of a new problem.

After missing an appointment and arriving at work about 3 hours late, I realized my phone stopped reproducing sound and vibrating when an alarm was fired. At first I thought I put the alarm incorrectly but then I verified that the alarm was not working. I still got a popup with the alarm message, but no sound or vibration.

So... Time to debug the problem. After searching a lot, I couldn't find anybody with my same problem, I found similar, but not the same, so I decided to report a bug. I got a very fast but useless response. Great!

Making long story short, I finally found some IRC channels and mailing lists where I could find a more opensourceish support that the one provided in the forums and bugzilla. So I'm happy I finally found a place where you can talk to actual developers.

I commented my problem and just after a very trivial but extremely useful suggestion (installing syslogd), I could trace the origin or the problem and fix it (I just love you strace!).

I also had another problem, suddenly the skype calls stopped working. Again the syslog helped a lot. Unfortunately I didn't save the exact syslog error message, but it was something like:

GStreamer - Could not convert static caps "!`phmcadion/x-rtp,
media=(string)video, payload=(int)[ 96, 127 ], clock-rate=(int)[ 1,
2147483647 ], encoding-name=(string)MP4V-ES"

As the MIME TYPE looked like garbage, I just grep(1)ed the filesystem searching for that string, and I found some binary file at /home/user/.gstreamer-0.10/registry.arm.bin. I backed up the file, remove it, and everything started working again (the file was recreated but with a very different content).

I have no idea how the symlink or the gstreamer file got broken, except maybe because of the unexpected reboot because of the broken batterypatch, but still, is really strange.

Anyway... Lessons learned:

• Maemo (Nokia) bugzilla is useless for getting help
• Install syslogd to debug Nokia N900 problems
• The maemo developers mailing list is your friend

Conclusion: Reflash my ass!

I bought a Nokia N900 recently, a great toy if you like to have a phone with a Linux distribution that uses dpkg as package manager :)

Of course you can use it as an end user, and never find out, but as the geek I am, I had to hack it, and use the devel package repositories. Of course, with that comes the problems (and the fun! :D).

The last update of the batterypatch package came with a weird feature. The device rebooted itself each time it starts, leaving it in a restart loop that rendered the device unusable.

Searching for valuable information was not easy (thanks forums! You SUCK at organizing information... I miss mailing lists).

Anyway, I hope I can save some work to someone if you get in a similar situation, so you don't have to waste ours searching the Maemo Forums.

First you will need a tool to flash the phone (it can do other things besides flashing it, I used the maemo_flasher-3.5_2.5.2.2_i386.deb file). You can also check some instructions on how to load a (very) basic rescue image (from Meego). The good thing is this image is an initrd that's loaded in MEMORY, so you don't loose anything if you tried, the device goes to it's previous state (broken in my case :P) after a reboot.

What this image can do is put the device in USB mass storage mode (the embedded MMC -eMMC- and the external MMC). I've done this to backup my eMMC data, which holds the MyDocs vfat partition and the 2 GiB ext3 partition used to install optional software. You can also put the device in USB networking mode, you can get a shell console (and reboot/power off the device), but I found that pretty useless (because you don't have any useful tools, the backlit is not turned on, so is really hard to see anything, and because the kayboard doesn't have the function key mapped, so you can't even write a "/").

The bad thing about this image, is you can't access to the root filesystem (wich is stored in another NAND 256MiB memory). I wanted to access it for 2 reasons. First, I wanted to edit some files that the batterypatch program created to see if that fixed the rebooting problem. And if now, I wanted to make a backup of the rootfs so I didn't loose most of my customizations and installed software.

I first found that a way to access the rootfs was to install Meego in a uSD memory, but for that I needed a 4GiB uSD. Also it looked like too much work, it has to be something battery and easier to just mount the rootfs and play around.

And I finally found it. It was the hardest thing to found, that's why I not only passing you the original link, I'm also hosting my own copy because I have the feeling it can disappear any time! :P

This image let's you do all the same the other image can, but it turns on the backlit, it has better support for the keyboard (you can type a "/") and it can mount the UBI root filesystem. Even more, it comes with a telnet daemon, so you can even do the rescue work remotely using USB networking ;)

You can see the instructions for some of the tasks, but here is how I did to be able to log in using telnet, which is not documented elsewhere that I know off. Once you have your image loaded:

1. You have to activate the USB networking in the device: /rescueOS/usbnetworking-enable.sh

2. Configure your host PC to assign an IP to usb0: sudo ip a add 192.168.2.14/24 dev usb0 && sudo ip link set usb0 up

3. Start the telnet daemon in the device: telnetd

4. I couldn't find out the root password, and since the initrd root filesystem is read-only, so I did this to change the root password:

   cp -a /etc /run/
   mount --bind /run/etc /etc
   passwd
   

   Now type the new root password.

5. That's it, log in via telnet from the host PC: telnet 192.168.2.15 and have fun!

With this I just could edit the broken files and saved the device without even needing to reflash it, but if you're not so lucky, you can just backup the root filesystem and reflash using this instructions (I didn't tested them, but seems pretty official).

Now I should probably have to try the recovery-boot package, if it works well it might be even easier to rescue the phone using that ;)

Graffiti Markup Language is...

An universal, XML based, open file format designed to store graffiti motion data (x and y coordinates and time). The format is designed to maximize readability and ease of implementation, even for hobbyist programmers, artists and graffiti writers. Popular applications currently implementing GML include Graffiti Analysis and EyeWriter. Beyond storing data, a main goal of GML is to spark interest surrounding the importance (and fun) of open data and introduce open source collaborations to new communities. GML is intended to be a simple bridge between ink and code, promoting collaborations between graffiti writers and hackers.

An probably the funniest part:

GML is today’s new digital standard for tomorrow’s vandals.

Interesting music video by The Black Keys shaped as a movie trailer.

Kind of Tarantinish / Rodriguish.

Tilt is a Firefox extension that lets you visualize any web page DOM tree in 3D.

Via Mozilla hacks.

Fragment of Law and Oracle episode (S06E17/6ACV16):

10 canciones delirantes con voces extrañas

[Grooveshark murió y con él esta lista]

Harvey Krumpet (es) by Adam Elliot

(or you can see it in 1 part only in English without Spanish subtitles here)

My cheap bike front light is not very water resistant...

I'm moving to Berlin on October to work at Sociomantic Labs. They use D + Tango with the GC I did as my thesis so I probably get the chance to improve the GC further!

Jetz, soll ich Deutsch lernen!

I was just listening to Dopádromo from Babasónicos and thinking: what a good album!

The Architecture of Open Source Applications

Architects look at thousands of buildings during their training, and study critiques of those buildings written by masters. In contrast, most software developers only ever get to know a handful of large programs well—usually programs they wrote themselves—and never study the great programs of history. As a result, they repeat one another's mistakes rather than building on one another's successes.

This book's goal is to change that. In it, the authors of twenty-five open source applications explain how their software is structured, and why. What are each program's major components? How do they interact? And what did their builders learn during their development? In answering these questions, the contributors to this book provide unique insights into how they think.

If you are a junior developer, and want to learn how your more experienced colleagues think, this book is the place to start. If you are an intermediate or senior developer, and want to see how your peers have solved hard design problems, this book can help you too.

I hope I can find the time to read this (at least some chapters).

Thank you Garrett, you saved my bike! =D

Today I got, after looking for one for several months, my derailleur hanger!

Raleigh (both Argentina and USA) were completely unhelpful, so I got to find it somewhere else, and finally found one at bicyclederailleurhangers.com (mine is #22 =).

I hope tomorrow I can finally fix my bike.

/me happy!

Lime total.

Fujitsu I hate you. I usually use an alias when putting my e-mail in some company form, using the feature most MTAs have to use something like myrealuser-<somealias>@example.com.

Lately I've been receiving tons of spam from luca-fujitsu@... and the kind of spam my Bogofilter have a hard time to swallow, so it's becoming really annoying. I'll just see how to make luca-fujitsu@... and invalid e-mail and reject all mails delivered to it.

Fuck off Fujitsu!

I've updated my Mutt Debian package with the NNTP patch to the latest Debian Mutt package.

This release is to bring just the regular bugfixing round from Debian.

If you have Debian testing/unstable and amd64 or i386 arch, just download and install the provided packages.

For other setups, here are the quick (copy&paste) instructions:

ver=1.5.21
deb_ver=$ver-5nntp1
url=https://llucax.com.nyud.net/proj/mutt-nntp-debian/files/latest
wget $url/mutt_$deb_ver.dsc $url/mutt_$deb_ver.diff.gz \
   http://ftp.de.debian.org/debian/pool/main/m/mutt/mutt_$ver.orig.tar.gz
sudo apt-get build-dep mutt
dpkg-source -x mutt_$deb_ver.dsc
cd mutt-$ver
dpkg-buildpackage -rfakeroot
# install any missing packages reported by dpkg-buildpackage and try again
cd ..
sudo dpkg -i mutt_${deb_ver}_*.deb mutt-patched_${deb_ver}_*.deb

See the project page for more details.

New section, let's see for how long I can keep this up =)

The idea is to make a compilation of 10 songs of (somehow) unknown music (and/or bands) picking up a theme. At first I'll try to pick only local (Argentinian) bands, but maybe in the future I can go more global.

This time, the theme is songs that have, in one way or another, some native rhythm.

[Groveshark died, and with it this list]

Quoting again =P (from Wikipedia):

In their March 2011 meeting, the ISO/IEC JTC1/SC22/WG21 C++ Standards Committee voted C++0x (N3290) to Final Draft International Standard (FDIS) status. This means that this final draft, dated 11 April 2011, is ready for review and approval by the ISO; the final specification is expected to be published sometime in mid-2011. To be able to finish closer to schedule, the Committee decided to focus its efforts on the solutions introduced up until 2006 and ignore newer proposals.

I still don't know if this is good or bad news. Probably both.

I'm too lazy lately, so I will just quote the Upstream Tracker site:

Backward binary compatibility testing for C and C++ shared libraries on x86

This service is aimed on analyzing of the C and C++ libraries evolution. It is looking for new releases of various libraries and checking them for backward binary compatibility. The web-service is generally intended for operating system maintainers to help in updating libraries and for software developers interested in ensuring backward binary compatibility of the API.

A really nice service. They are monitoring 286 libraries (at the time of writing) and you can add more for free.

Bitcoin is one of the most subversive ideas I ever read, it's as scary as exciting in how it could change the world economy dynamics if it works.

Bitcoin is (quoting WeUseCoins.com):

Decentralized

Bitcoin is the first digital currency that is completely distributed. The network is made up of users like yourself so no bank or payment processor is required between you and whoever you're trading with. This decentralization is the basis for Bitcoin's security and freedom.

Worldwide

Your Bitcoins can be accessed from anywhere with an Internet connection. Anybody can start mining, buying, selling or accepting Bitcoins regardless of their location.

No small print

If you have Bitcoins, you can send them to anyone else with a Bitcoin address. There are no limits, no special rules to follow or forms to fill out.

More complex types of transactions can be built on top of Bitcoin as well, but sometimes you just want to send money from A to B without worrying about limits and policies.

Very low fees

Currently you can send Bitcoin transactions for free. However, a fee on the order of 1 bitcent will eventually be necessary for your transaction to be processed more quickly. Miners compete on fees, which ensures that they will always stay low in the long run. More on transaction fees (Bitcoin Wiki).

Own your money!

You don't have to be a criminal to wake up one day and find your account has been frozen. Rules vary from place to place, but in most jurisdictions accounts may be frozen by credit card collection agencies, by a spouse filing for divorce, by mistake or for terms of service violations.

In contrast, Bitcoins are like cash - seizing them requires access to your private keys, which could be placed on a USB stick, thereby enjoying the full legal and practical protections of physical property.

Here is a video, if you are too lazy to read:

If you want some more detailed information, there is a paper describing the technical side of the project (which I read and didn't fully understand, to be honest).

You have to add bitcoin mining to the equation. Which is not very well explained there. Bitcoin mining is a business, just like gold mining is. You need resources to do it, and if you don't do it efficiently, you'll loose money (the electricity and hardware cost will supersede what you're earning).

Quoting again:

The mining difficulty expresses how much harder the current block is to generate compared to the first block. So a difficulty of 70000 means to generate the current block you have to do 70000 times more work than Satoshi had to do generating the first block. Though be fair though, back then mining was a lot slower and less optimized.

The difficulty changes every 2016 blocks. The network tries to change it such that 2016 blocks at the current global network processing power take about 14 days. That's why, when the network power rises, the difficulty rises as well.

El Cielo de Canarias / Canary sky

Just a stupid idea...

How to use your fridge as a whiteboard

First, get a fridge, then, get a whiteboard marker

Find a hidden spot and write something, then try to erase it immediately

If it didn't work, get another fridge

Otherwise, write again in the hidden spot, but don't erase it yet, wait a couple of hours, then erase it

If it didn't work, get another fridge

If it worked, congratulations, your fridge can be used as a whiteboard

Write on it with the marker, as I am doing (doodle)

Optionally, you can buy some cool magnets (like these)

Glue a couple to your marker (add a protection so they don't scratch the fridge (if you care enough)

So you can leave the marker right on the fridge, ready to be used

Enjoy your fridge, bye =)

I always loved Studio Ghibli's movies, and mostly because I found them so imaginative, I love the fantasy worlds they created.

But then, I saw this video, and it made me think that maybe, Sen to Chihiro (Spirited Away) is really a documentary...

I love The RSA animated series of videos. Here are a couple of random ones:

Enjoy!

I've updated my Mutt Debian package with the NNTP patch to the latest Debian Mutt package.

If you have downloaded the previous version, you probably noted an extremely annoying bug, which is fixed in this new package, so I'm sure you want to upgrade =)

If you have Debian testing/unstable and amd64 or i386 arch, just download and install the provided packages.

For other setups, here are the quick (copy&paste) instructions:

ver=1.5.21
deb_ver=$ver-4nntp1
url=https://llucax.com.nyud.net/proj/mutt-nntp-debian/files/latest
wget $url/mutt_$deb_ver.dsc $url/mutt_$deb_ver.diff.gz \
   http://ftp.de.debian.org/debian/pool/main/m/mutt/mutt_$ver.orig.tar.gz
sudo apt-get build-dep mutt
dpkg-source -x mutt_$deb_ver.dsc
cd mutt-$ver
dpkg-buildpackage -rfakeroot
# install any missing packages reported by dpkg-buildpackage and try again
cd ..
sudo dpkg -i mutt_${deb_ver}_*.deb mutt-patched_${deb_ver}_*.deb

See the project page for more details.

Today, in the lands of YouTube, is First Orbit, the movie...

A free movie for you to download and share, created to celebrate the first 50 years of human space flight.

A real time recreation of Yuri Gagarin's pioneering first orbit, shot entirely in space from on board the International Space Station. The film combines this new footage with Gagarin's original mission audio and a new musical score by composer Philip Sheppard.

What I like the most is, it's the Russian side of the coin, and when it come to the space race, we usually only see the USA side of it (as with many many other historical events).

Here is the entire movie:

According to Numbeo website:

Numbeo is the largest free Internet database about cost of living worldwide!

Numbeo is the largest free Internet database about worldwide housing indicators!

It's a really nice site to see how the prices compare between countries, and other type of indicators for travelers or curious =)

Algunas notas (English speakers ignore this, is not important):

• GEBA (o mejor dicho los maquinistas de TBA) apesta; el tren no molesta tanto pero los pelotudos de los maquinistas se la pasaba tocando bocina en busca de 15 segundos de (mala) fama.
• Que exista campo y campo VIP apesta. Odio, pero ODIO ODIO al que se le ocurrió esa nefasta idea (cánticos del estilo "¡Sacá la valla la putá que lo parió!" no faltaron). Sí, a vos geniecillo del marketing desconocido, te maldigo y espero que te agarres alguna enfermedad fea fea.
• El show fue una fiesta.

The Flaming Lips @ GEBA (2011-04-05) (1)

2011-04-05 21:20:04 ART

[ 1/80s f/4.0 iso400 11.3 mm Cloudy Manual ]

The Flaming Lips @ GEBA (2011-04-05) (2)

2011-04-05 21:22:33 ART

[ 1/100s f/4.0 iso800 8.8 mm Cloudy Manual ]

The Flaming Lips @ GEBA (2011-04-05) (3)

2011-04-05 21:25:17 ART

[ 1/100s f/5.0 iso400 36.4 mm Cloudy Manual ]

The Flaming Lips @ GEBA (2011-04-05) (4)

2011-04-05 21:31:27 ART

[ 1/80s f/5.6 iso400 57.6 mm Cloudy Manual ]

The Flaming Lips @ GEBA (2011-04-05) (5)

2011-04-05 21:47:52 ART

[ 1/80s f/5.0 iso800 19.1 mm Cloudy Manual ]

The Flaming Lips @ GEBA (2011-04-05) (6)

2011-04-05 22:00:06 ART

[ 1/160s f/5.9 iso1600 70.0 mm Cloudy Manual ]

The Flaming Lips @ GEBA (2011-04-05) (7)

2011-04-05 22:04:44 ART

[ 1/40s f/5.0 iso1600 24.7 mm Cloudy Manual ]

The Flaming Lips @ GEBA (2011-04-05) (8)

2011-04-05 22:05:01 ART

[ 1/40s f/5.0 iso1600 24.7 mm Cloudy Manual ]

The Flaming Lips @ GEBA (2011-04-05) (9)

2011-04-05 22:18:43 ART

[ 1/80s f/5.6 iso1600 50.4 mm Cloudy Manual ]

The Flaming Lips @ GEBA (2011-04-05) (10)

2011-04-05 22:50:41 ART

[ 1/250s f/4.0 iso1600 11.5 mm Cloudy Manual ]

Massacre obrando de soporte de los Flaming Lips.

Massacre @ GEBA (2011-04-05) (1)

2011-04-05 20:24:20 ART

[ 1/50s f/5.9 iso1600 70.0 mm Auto Manual ]

Massacre @ GEBA (2011-04-05) (2)

2011-04-05 20:25:33 ART

[ 1/50s f/5.9 iso1600 70.0 mm Auto Manual ]

There are no words to describe this...

This happened about a month ago in a Critical Mass in Porto Alegre, Brazil.

Is The Canterbury Distribution the first big joke from April Fools's Day?

I'm sure it is =)

Whoa!

I just want to say: WHAT AN AMAZING ALBUM

It's been a while since I listened to an album that made me want to say that, and Let England Shake by Pj Harvey did it (and I'm exactly a PJ Harvey fan). Both music and lyrics are excellent.

Also, there are videos available for almost all the songs in the album, and they are all beautiful. Here is the video for The Colour Of The Earth HD. Watch it!

(vía albertito =P)

Well, I usually only removed those nice and powerful magnets, but this is going one step further at the time of finding a use for a broken HDD...

Martes 15 de Marzo, Complejo Cultural (Teatro) 25 de Mayo.

Wander Wildner y sus Comancheros (1)

2011-03-15 22:00:34 ART

[ 1/20s f/5.9 iso400 70.0 mm Cloudy Manual ]

Wander Wildner y sus Comancheros (2)

2011-03-15 21:23:28 ART

[ 1/25s f/5.9 iso400 70.0 mm Cloudy Manual ]

Wander Wildner y sus Comancheros (3)

2011-03-15 21:38:49 ART

[ 1/15s f/5.9 iso400 70.0 mm Cloudy Manual ]

Wander Wildner y sus Comancheros (4)

2011-03-15 21:48:45 ART

[ 1/20s f/5.0 iso400 37.2 mm Cloudy Manual ]

Here is the winning dunk of the 23th Argentinian's basketball league All Stars dunk contest by DeCorey Young:

In case you didn't notice, he was wearing Messi's T-shirt =)

You know how Wikipedia is... One thing led to another and I ended up reading about gecko adhesion in one of its references:

How do geckos make use of the smallest of intermolecular forces to climb walls? Tiny hairs derived from the keratin in the skin on their feet create a large, compliant surface area which makes intimate contact with the substrate. One Tokay gecko (Gekko gecko) may possess several millions of these hairs, called setae. Each seta is in turn subdivided into 100-1000 smaller flattened tips, called spatulae.

Why van der Waals? Although they are the weakest type of intermolecular force, they are ubiquitous and occur between all types of surfaces. This means that the key to dry adhesion is the shape or geometry of the adhesive, rather than the chemistry. Other insects which stick by secretions (e.g. ants, beetles, flies, etc.) are much more picky about what types of surfaces they stick to. Geckos can stick to any surface, with the exception of Teflon, which was specifically engineered to prevent even van der Waals adhesion.

Even more, there is some research on synthesizing artificial gecko adhesives. Very interesting...

Lazy Boy by Les Mentettes Orchestra.

Yai! Finally my CDGC patches has been applied to Tango [1] [2] [3]. CDGC will not be the default Tango GC for now, because it needs some real testing first (and fixing a race when using weak references). So, please, please, do try it, is as simple as compiling from the sources adding a new option to bob: -g=cdgc and then manually installing Tango.

Please, don't forget to report any bugs or problems.

Thanks!

I've updated my Mutt Debian package with the NNTP patch to the latest Debian Mutt package.

A couple of extra news:

• I added a new Git repository so you can download the sources via Git.
• The Debian revision is renamed to NnntpM (instead of the old NlucaM).

If you have Debian testing/unstable and amd64 or i386 arch, just download and install the provided packages.

For other setups, here are the quick (copy&paste) instructions:

ver=1.5.21
deb_ver=$ver-2nntp1
url=https://llucax.com.nyud.net/proj/mutt-nntp-debian/files/latest
wget $url/mutt_$deb_ver.dsc $url/mutt_$deb_ver.diff.gz \
   http://ftp.de.debian.org/debian/pool/main/m/mutt/mutt_$ver.orig.tar.gz
sudo apt-get build-dep mutt
dpkg-source -x mutt_$deb_ver.dsc
cd mutt-$ver
dpkg-buildpackage -rfakeroot
# install any missing packages reported by dpkg-buildpackage and try again
cd ..
sudo dpkg -i mutt_${deb_ver}_*.deb mutt-patched_${deb_ver}_*.deb

See the project page for more details.

The 2010 LLVM Developers' Meeting was almost 2 months ago, and this time Apple employees videos are available.

I saw the libclang talk video and it was really interesting, it made me want to play with clang. I wonder if there are Python bindings to it, is so much easier to try things out using Python ;)

Marcello Perathoner, from Project Gutenberg, has just announced that they are considering using ReStructuredText as master format. In his own words:

Project Gutenberg (PG) is a volunteer effort to digitize and make available to everybody books whose copyright has expired. We OCR the books and proofread them and make them available in a variety of formats.

PG has been evaluating RST as a master format to generate HTML, EPUB, Kindle, PDF and plain text formats.

In that course I have written some patches to RST. I now want to discuss them and maybe improve my ways. If you see fit, my patches could be added to the RST core.

It's nice to see this two lovely projects playing together :)

Finally, I defended my thesis last Monday and now I'm officially (well, not really, the diploma takes about a year to be emitted) an Ingeniero en Informática (something like a Informatics Engineer). I hope I can get some free time now to polish the rough edges of the collector (fix the weakrefs for example) so it can be finally merged into Tango.

It looks like Walter is willing to (finally) give the copyright assignment of the DMD front-end to the FSF, so GDC can have a chance at being merged to the official GCC distribution.

Let's see how this turns out...

Sean Kelly just created a new experimental branch in Druntime with CDGC as the GC for D2. The new branch is completely untested though, so only people wanting to help testing should try it out (which will be very appreciated).

I've posted a small set of patches to integrate CDGC into Tango. If you want to try them out, just follow the simple 3 steps described in the ticket.

Please, let me know how it went if you do.

Here are some details on how to try CDGC, as it needs a very particular setup, specially due to DMD not having precise heap scanning integrated yet.

Here are the steps (in some kind of literate scripting, you can copy&paste to a console ;)

# You probably want to do all this mess in some subdirectory :)
mkdir cdgc-test
cd cdgc-test

# First, checkout the repositories.
git clone git://git.llucax.com/software/dgc/cdgc.git
# If you have problems with git:// URLs, try HTTP:
# git clone https://git.llucax.com/r/software/dgc/cdgc.git
svn co http://svn.dsource.org/projects/tango/tags/releases/0.99.9 tango

# DMD doesn't care much (as usual) about tags, so you have to use -r to
# checkout the 1.063 revision (you might be good with the latest revision
# too).
svn co -r613 http://svn.dsource.org/projects/dmd/branches/dmd-1.x dmd

# Now we have to do some patching, let's start with Tango (only patch 3 is
# *really* necessary, but the others won't hurt).
cd tango
for p in 0001-Fixes-to-be-able-to-parse-the-code-with-Dil.patch \
         0002-Use-the-mutexattr-when-initializing-the-mutex.patch \
         0003-Add-precise-heap-scanning-support.patch \
         0004-Use-the-right-attributes-when-appending-to-an-empty-.patch
do
   wget -O- "https://llucax.com/blog/posts/2010/10/10-trying-cdgc-howto/$p" |
         patch -p1
done
cd ..

# Now let's go to DMD
cd dmd
p=0001-Create-pointer-map-bitmask-to-allow-precise-heap-sca.patch
wget -O- "https://llucax.com/blog/posts/2010/10/10-trying-cdgc-howto/$p" |
      patch -p1

# Since we are in the DMD repo, let's compile it (you may want to add -jN if
# you have N CPUs to speed up things a little).
make -C src -f linux.mak
cd ..

# Good, now we have to wire Tango and CDGC together, just create a symbolic
# link:
cd tango
ln -s ../../../../../cdgc/rt/gc/cdgc tango/core/rt/gc/

# Since I don't know very well the Tango build system, I did a Makefile of my
# own to compile it, so just grab it and compile Tango with it. It will use
# the DMD you just compiled and will compile CDGC by default (you can change
# it via the GC Make variable, for example: make GC=basic to compile Tango
# with the basic GC). The library will be written to obj/libtango-$GC.a, so
# you can have both CDGB and the basic collector easily at hand):
wget https://llucax.com/blog/posts/2010/10/10-trying-cdgc-howto/Makefile
make # Again add -jN if you have N CPUs to make a little faster

# Now all you need now is a decent dmd.conf to put it all together:
cd ..
echo "[Environment]" > dmd/src/dmd.conf
echo -n "DFLAGS=-I$PWD/tango -L-L$PWD/tango/obj " >> dmd/src/dmd.conf
echo -n "-defaultlib=tango-cdgc " >> dmd/src/dmd.conf
echo "-debuglib=tango-cdgc -version=Tango" >> dmd/src/dmd.conf

# Finally, try a Hello World:
cat <<EOT > hello.d
import  tango.io.Console;

void main()
{
   Cout("Hello, World").newline;
}
EOT
dmd/src/dmd -run hello.d

# If you don't trust me and you want to be completely sure you have CDGC
# running, try the collect_stats_file option to generate a log of the
# collections:
D_GC_OPTS=collect_stats_file=log dmd/src/dmd -run hello.d
cat log

Done!

If you want to make this DMD the default, just add dmd/src to the PATH environment variable or do a proper installation ;)

Let me know if you hit any problem...

Just in case you are a Bacap user (and a lazy one, that doesn't want to subscribe to the repository RSS feed :), you might be interested in a few recent updates:

• Make pinging hosts configurable (a little older :).
• Add FORCE_SYNC option.
• Let rsync do the hard linking.
• Don't provide any default options for ssh.
• Add a Tips section to the README (you can see the tips in the updated website).
• Fix console error reporting when using a LOG_FILE.

Enjoy!

I'm sorry about the quick and uninformative post, but I've been almost 2 weeks without Internet and I have to finish the first complete draft of my thesis in a little more than a week, so I don't have much time to write here.

The thing is, to avoid the nasty effect of memory usage being too high for certain programs when using eager allocation, I've made the GC minimize the heap more often. Even when some test are still a little slower with CDGC, but that's only for tests that only stress the GC without doing any actual work, so I think it's OK, in that cases the extra overhead of being concurrent is bigger than the gain (which is inexistent, because there is nothing to do in parallel with the collector).

Finally, I've implemented early collection, which didn't proved very useful, and tried to keep a better occupancy factor of the heap with the new min_free option, without much success either (it looks like the real winner was eager allocation).

I'm sorry I don't have time to show you some graphs this time. Of course the work is not really finished, there are plenty of things to be done still, but I think the GC have come to a point where it can be really useful, and I have to finish my thesis :)

After I'm done, I hope I can work on integrating the GC in Tango and/or Druntime (where there is already a first approach done by Sean Kelly).

Huy, tal vez estuve un poco fuerte con el título... O tal vez no.

Hace 13 días que estoy sin teléfono (y por consiguiente Internet, porque tengo ADSL). En realidad vengo con problemas con el teléfono desde junio; los días de lluvia se escuchaba con ruido hasta cortarse (pero extrañamente se cortaba el teléfono pero no Internet) y luego volvía solo.

La fantástica empresa, tan apreciada y conocida por todos, llamada Telecom Argentina funciona tan bien, que llamás al 114 (bah, llama algún conocido porque vos no tenés teléfono) y te atiende un contestador y te dice que tomó el reclamo. Nadie habla de tiempos, nadie habla de nada. Ni te gastes en buscar algún lugar físico de atención al cliente porque al menos en la factura no figura nada, es una empresa fantasma.

Llamé varias veces por el tema de los cortes por lluvia pero nada, solo te amenazan con que cobran la visita si es problema interno de tu casa así que te dicen que te fijes bien que en la entrada del cable a tu casa ande todo bien. El problema es cuando nunca te dijeron donde pusieron la mágica cajita de Telecom que divide tu reino del de ellos. Tener un techo de tejas altísimo no ayuda, porque en general eso está en la terraza.

Luego de probar de todo, llamé y pedí un técnico, cagándome en las amenazas. Pero no, claro, ellos no pueden concertar una cita para un día y hora... Bueno, nadie te dice hora, pero ellos ni siquiera día te dicen. La mecánica es así, te mandan a alguien cuando ellos quieren, si estás bien, y si no recién ahí ven si combinan algún día. ¡Súper eficiente! Y no es que tampoco tarden un día en venir, perder una visita no es poca cosa.

La cosa es que un día cayeron tipo 8 de la mañana, y estaba más dormido que alguien que no está tan tan dormido como yo estaba, que era bastante mucho, y no escuché el timbre. Pero fueron lo suficientemente amables como para llamarme al celular hasta despertarme, a lo que después de regañar un poco por ser unos forros por caer cuando se le canta le dije que ahora los atendía, que me banquen 10 minutos que me levantaba. Ahhhh, la amabilidad de Telecom, tocaron impacientemente el timbre un par de veces más mientras me cambiaba y cuando salí ya se habían ido. Llamé a Telecom para ver que onda, y obviamente nadie sabe nada, es la compañía con peor comunicación interna del mundo, o con peor atención del mundo (me inclino por la segunda). Además, tampoco nadie te dice nada sobre plazos, le pregunté explícitamente si podían tardar un día o una semana o un mes y el muy descarado me dijo "sí". Todo es posible en el maravilloso mundo de Telecom.

Finalmente me llamaron para concertar la entrevista pero el día anterior había llovido y no había pasado nada así que la cancelé (porque lo último que faltaba era que cuando vengan no puedan encontrar el problema y me quieran cobrar diciendo que era problema mío).

Y bueno, así pasaron los días y se hizo 14 de septiembre y el teléfono se cortó a las 16:00hs para nunca más volver, así que llamé de nuevo. El viernes 17, con la velocidad de una babosa sedada, llegó el técnico (de nuevo sin avisar y que me enganchó de pedo antes de salir al laburo) y quiso entrar a la casa, pero le advertí que no había nada que ver adentro, que Telecom a mí nunca me dijo ni donde había puesto la caja. Así que totalmente perdido, se fue a la puerta a verificar una caja externa, con su escalerita de metro y medio que apenas le alcanzó para llegar a esa caja (a mi techo de unos 5 o 6 metros mucho no podía llegar). Y ahí había tono. Con cara de preocupado me dijo que tenía que pedir a otra gente para que lo arregle, a lo que, con cara de resignado, le pregunté "y cuando van a venir", y me dijo, siguiendo firme la política de Telecom "ni idea, ahora con la lluvia viste como es, aparecen todas estas cosas y hay poco personal" a lo que le imploro con cara de desesperado que al menos me de un mínimo o máximo, y me dice que mínimo una semana... ¡Groso!

Llamé de nuevo a Telecom, al área comercial porque en reparaciones te atiende un contestador que dice "su línea ya está en reparaciones" y te corta, y me dicen que ellos no saben nada, que llame a reparaciones. Unos FE NO ME NOS.

El fin de semana, de pedo mirando por la ventana, noté que de la pared del fondo salía un cable, que iba todo por la medianera del fondo hasta el jardín de la vecina, que colgaba largamente hasta subir a su tejado, pero que por la mitad tenía un hermoso empalme, sin siquiera un poco de cinta. Así a lo machote, al aire libre.

Me brillaron los ojitos un poco, pero la indignación casi me mata. Haciéndola corta, le pedí escalera larga a mis viejos prestada, permiso a la vecina y esta tarde por fin pude poner mis manos en él, y voilá! Tengo teléfono/Internet luego de 13 días de corte, gracias a mi alpedismo de mirar por la ventana, mi vecina que se copó, y mis viejos que se ultra-coparon en traerme la escalera. NO gracias a Telecom. Ahora tendré que seguir peleándome para que por un lado me descuenten los días sin servicio, por otro me paguen la multa del doble del abono proporcional por cada día sin servicio por no arreglarlo dentro de los 3 días y finalmente porque vengan a hacer una instalación como la gente, no ese mamamarracho que hicieron de cables colgados y empalmes (hay 2 cables más iguales colgando en el jardín de la vecina).

En fin, están advertidos, si tienen problemas con Telecom, empiecen a buscar la forma de arreglárselos ustedes mismos...

All the millions of users of mutest that were loosing their minds trying to figure out why mkmutest was choking in 64 bits OSs, your suffering days are over, since I added 64 bits OSs (Linux) support.

Yeah, a great day of pure joy for the entire humanity!

Finally, I got the first version of CDGC with truly concurrent garbage collection, in the sense that all the threads of the mutator (the program itself) can run in parallel with the collector (well, only the mark phase to be honest :).

You might want to read a previous post about CDGC where I achieved some sort of concurrency by making only the stop-the-world time very short, but the thread that triggered the collection (and any other thread needing any GC service) had to wait until the collection finishes. The thread that triggered the collection needed to wait for the collection to finish to fulfill the memory allocation request (it was triggered because the memory was exhausted), while any other thread needing any GC service needed to acquire the global GC lock (damn global GC lock!).

To avoid this issue, I took a simple approach that I call eager allocation, consisting on spawn the mark phase concurrently but allocating a new memory pool to be able to fulfill the memory request instantly. Doing so, not only the thread that triggered the collection can keep going without waiting the collection to finish, the global GC lock is released and any other thread can use any GC service, and even allocate more memory, since a new pool was allocated.

If the memory is exhausted again before the collection finishes, a new pool is allocated, so everything can keep running. The obvious (bad) consequence of this is potential memory bloat. Since the memory usage is minimized from time to time, this effect should not be too harmful though, but let's see the results, there are plenty of things to analyze from them (a lot not even related to concurrency).

First, a couple of comments about the plots:

• Times of Dil are multiplied by a factor of 0.1 in all the plots, times of rnddata are too, but only in the pause time and stop-the-world plots. This is only to make the plots more readable.
• The unreadable labels rotated 45 degrees say: stw, fork and ea. Those stand for Stop-the-world (the basic collector), fork only (concurrent but without eager allocation) and eager allocation respectively. You can click on the images to see a little more readable SVG version.
• The plots are for one CPU-only because using more CPUs doesn't change much (for these plots).
• The times were taken from a single run, unlike the total run time plots I usually post. Since a single run have multiple collections, the information about min, max, average and standard deviation still applies for the single run.
• Stop-the-world time is the time no mutator thread can run. This is not related to the global GC lock, is time the threads are really really paused (this is even necessary for the forking GC to take a snapshot of threads CPU registers and stacks). So, the time no mutator thread can do any useful work might be much bigger than this time, because the GC lock. This time is what I call Pause time. The maximum pause time is probably the most important variable for a GC that tries to minimize pauses, like this one. Is the maximum time a program will stay totally unresponsive (important for a server, a GUI application, a game or any interactive application).

The stop-the-world time is reduced so much that you can hardly see the times of the fork and ea configuration. It's reduced in all tests by a big margin, except for mcore and the bigarr. For the former it was even increased a little, for the later it was reduced but very little (but only for the ea* configuration, so it might be a bad measure). This is really measuring the Linux fork() time. When the program manages so little data that the mark phase itself is so fast that's faster than a fork(), this is what happens. The good news is, the pause times are small enough for those cases, so no harm is done (except from adding a little more total run time to the program).

Note the Dil maximum stop-the-world time, it's 0.2 seconds, looks pretty big, uh? Well, now remember that this time was multiplied by 0.1, the real maximum stop-the-world for Dil is 2 seconds, and remember this is the minimum amount of time the program is unresponsive! Thank god it's not an interactive application :)

Time to take a look to the real pause time:

OK, this is a little more confusing... The only strong pattern is that pause time is not changed (much) between the swt and fork configurations. This seems to make sense, as both configurations must wait for the whole collection to finish (I really don't know what's happening with the bh test).

For most tests (7), the pause time is much smaller for the ea configuration, 3 tests have much bigger times for it, one is bigger but similar (again mcore) and then is the weird case of bh. The 7 tests where the time is reduced are the ones that seems to make sense, that's what I was looking for, so let's see what's happening with the remaining 3, and for that, let's take a look at the amount of memory the program is using, to see if the memory bloat of allocating extra pools is significant.

See any relations between the plot and the table? I do. It looks like some programs are not being able to minimize the memory usage, and because of that, the sweep phase (which still have to run in a mutator thread, taking the global GC lock) is taking ages. An easy to try approach is to trigger the minimization of the memory usage not only at when big objects are allocated (like it is now), but that could lead to more mmap()/munmap()s than necessary. And there still problems with pools that are kept alive because a very small object is still alive, which is not solved by this.

So I think a more long term solution would be to introduce what I call early collection too. Meaning, trigger a collection before the memory is exhausted. That would be the next step in the CDGC.

Finally, let's take a look at the total run time of the test programs using the basic GC and CDGC with concurrent marking and eager allocation. This time, let's see what happens with 2 CPUs (and 25 runs):

Wow! It looks like this is getting really juicy (with exceptions, as usual :)! Dil time is reduced to about 1/3, voronoi is reduced to 1/10!!! Split and mcore have both their time considerably reduced, but that's because another small optimization (unrelated to what we are seeing today), so forget about those two. Same for rnddata, which is reduced because of precise heap scanning. But other tests increased its runtime, most notably bigarr takes almost double the time. Looking at the maximum heap size table, one can find some answers for this too. Another ugly side of early allocation.

For completeness, let's see what happens with the number of collections triggered during the program's life. Here is the previous table with this new data added:

See how the number of collections is practically reduced proportionally to the increase of the heap size. When the increase in size explodes, even when the number of collections is greatly reduced, the sweep time take over and the total run time is increased. Specially in those tests where the program is almost only using the GC (as in sbtree and bigarr). That's why I like the most Dil and voronoi as key tests, they do quite a lot of real work beside asking for memory or using other GC services.

This confirms that the performance gain is not strictly related to the added concurrency, but because of a nice (finally! :) side-effect of eager allocation: removing some pressure from the GC by increasing the heap size a little (Dil gets 3x boost in run time for as little as 1.16x of memory usage; voronoi gets 10x at the expense of almost doubling the heap, I think both are good trade-offs). This shows another weak point of the GC, sometimes the HEAP is way too tight, triggering a lot of collections, which leads to a lot of GC run time overhead. Nothing is done right now to keep a good heap occupancy ratio.

But is there any real speed (in total run time terms) improvement because of the added concurrency? Let's see the run time for 1 CPU:

It looks like there is, specially for my two favourite tests: both Dil and voronoi get a 30% speed boost! That's not bad, not bad at all...

If you want to try it, the repository has been updated with this last changes :). If you do, please let me know how it went.

The Wilderness Downtown is a new interactive film by Chris Milk (done in HTML5 as a Google Chrome Experiment), and the new video for the song We Used To Wait from Arcade Fire. Judging from this, and the Unstaged show it looks like they are willing to play with the new technologies.

I like the video, for some reason it reminds me of House Of Cards (maybe because Google was involved too). The downside is, it only works on Chrome / Chromium.

After a small (but important) step towards making the D GC truly concurrent (which is my main goal), I've been exploring the possibility of making the mark phase recursive instead of iterative (as it currently is).

The motivation is that the iterative algorithm makes several passes through the entire heap (it doesn't need to do the full job on each pass, it processes only the newly reachable nodes found in the previous iteration, but to look for that new reachable node it does have to iterate over the entire heap). The number of passes is the same as the connectivity graph depth, the best case is where all the heap is reachable through the root set, and the worse is when the heap is a single linked list. The recursive algorithm, on the other hand, needs only a single pass but, of course, it has the problem of potentially consuming a lot of stack space (again, the recurse depth is the same as the connectivity graph depth), so it's not paradise either.

To see how much of a problem is the recurse depth in reality, first I've implemented a fully recursive algorithm, and I found it is a real problem, since I had segmentation faults because the (8MiB by default in Linux) stack overflows. So I've implemented an hybrid approach, setting a (configurable) maximum recurse depth for the marking phase. If the maximum depth is reached, the recursion is stopped and nodes that should be scanned deeply than that are queued to scanned in the next iteration.

Here are some results showing how the total run time is affected by the maximum recursion depth:

The red dot is how the pure iterative algorithm currently performs (it's placed arbitrarily in the plot, as the X-axis doesn't make sense for it).

The results are not very conclusive. Even when the hybrid approach performs better for both Dil and Voronoi when the maximum depth is bigger than 75, the better depth is program specific. Both have its worse case when depth is 0, which makes sense, because is paying the extra complexity of the hybrid algorithm with using its power. As soon as we leave the 0 depth, a big drop is seen, for Voronoi big enough to outperform the purely iterative algorithm, but not for Dil, which matches it near 60 and clearly outperforms it at 100.

As usual, Voronoi challenges all logic, as the best depth is 31 (it was a consistent result among several runs). Between 20 and 50 there is not much variation (except for the magic number 31) but when going beyond that, it worsen slowly but constantly as the depth is increased.

Note that the plots might make the performance improvement look a little bigger than it really is. The best case scenario the gain is 7.5% for Voronoi and 3% for Dil (which is probably better measure for the real world). If I had to choose a default, I'll probably go with 100 because is where both get a performance gain and is still a small enough number to ensure no segmentation faults due to stack exhaustion is caused (only) by the recursiveness of the mark phase (I guess a value of 1000 would be reasonable too, but I'm a little scared of causing inexplicable, magical, mystery segfaults to users). Anyway, for a value of 100, the performance gain is about 1% and 3.5% for Dil and Voronoi respectively.

So I'm not really sure if I should merge this change or not. In the best case scenarios (which requires a work from the user to search for the better depth for its program), the performance gain is not exactly huge and for a reasonable default value is so little that I'm not convinced the extra complexity of the change (because it makes the marking algorithm a little more complex) worth it.

Feel free to leave your opinion (I would even appreciate it if you do :).

I once recommended The Yes Men, now their new movie, The Yes Men Fix The World is officially available for download in Vodo (HD and mobile versions included) via BitTorrent. There are subtitles for several languages (more are coming) and you can donate to The Yes Men so they can film their next movie.

Great mov(i)e!

I'm glad to announce that now, for the first time, CDGC means Concurrent D Garbage Collector, as I have my first (extremely raw and unoptimized) version of the concurrent GC running. And I have to say, I'm very excited and happy with the results from the very small benchmark I did.

The stop-the-world (pause) time was reduced by 2 orders of magnitude for the average, the standard deviation and, probably the more important, the maximum (these are the results for a single run, measuring the pause time for all the collections in that run). This is good news for people needing (soft) real-time in D, even when using the GC. Where the standard D GC have a pause time of 100ms, the CDGC have a pause time of 1ms.

The total run-time of the program was increased a little though, but not as much as the pause time was reduced. Only a 12% performance loss was measured, but this is just the first raw unoptimized version of the CDGC.

All this was measured with the voronoi benchmark, with -n 30000. Here are some plots:

Please note that the GC still has a global lock, so if 2 threads needs to allocate while the collection is running, both will be paused anyways (I have a couple of ideas on how to try to avoid that).

The idea about how to make the GC concurrent is based on the paper Nonintrusive Cloning Garbage Collector with Stock Operating System Support. I'm particularly excited by the results because the reduction of the pause time in the original paper were less than 1 order of magnitude better than their stop-the-world collector, so the preliminary results of the CDGC are much better than I expected.

You might already know about the new Arcade Fire album, The Suburbs.

Even when is a good album, I think is by far the less interesting / innovative / emotional from the band (at least from the musical point of view, I didn't payed much attention to the lyrics yet), but I still love their live performance, and I've came across the Unstaged series at the Madison Square Garden and directed by Terry Gilliam. The videos are available in YouTube in HD (many thanks to youtube-dl [*] that let me see the concert without suffering the f*cking Flash Player).

Here is Sprawl II (Mountains Beyond Mountains), my favourite song from the new album:

for i in h0DpBOYzXcQ 5pp3olGyku0 TtbrY6QrgPw 7vS5crSXJ6k GGdyG_83nX4 \
      DSykB-j_2UQ Q2T-ZAharmY 0L6ZFhZVOx0 CLjrQ3cwzJ4 RdYyYFymH-Y \
      sVW9usMRgNE bjhAP3V1o0w Q7fRvPjKDLs N9WR_eK0Rac bhwdd_01Ots \
      0ozdCLrTrtA diK1tZOXgDk 0ozdCLrTrtA aXuymDSGCko MQvZ4N1RfS8 \
      siFsdInZqC0 MXJi5_yThNY
do
   youtube-dl -f 37 -t "http://www.youtube.com/watch?v=$i"
done

The D compiler doesn't provide any information on the static data that the GC must scan, so the runtime/GC have to use OS-dependant tricks to get that information.

Right now, in Linux, the GC gets the static data to scan from the libc's variables __data_start and _end, from which are not much information floating around except for some e-mail from Hans Boehm to the binutils mainling list.

There is a lot of stuff in the static data that doesn't need to be scanned, most notably the TypeInfo, which is a great portion of the static data. C libraries static data, for example, would be scanned too, when it makes no sense to do so.

I noticed CDGC has more than double the static data the basic GC has, just because of TypeInfo (I use about 5 or so more types, one of them is a template, which makes the bloat bigger).

The voronoi test goes from 21KB to 26KB of static data when using CDGC.

It would be nice if the compiler could group all the static that must really be scanned (programs static variables) together and make its limits available to the GC. It would be even nicer to leave static variables that have no pointers out of that group, and even much more nicer to create a pointer map like the one in the patch for precise scanning to allow precise heap scanning. Then only the scan should be scanned in full conservative mode.

I reported a bug with this issue so it doesn't get lost.

After seeing some weird behaviours and how different benchmarks are more or less affected by changes like memory addresses returned by the OS or by different ways to store the type information pointer, I decided to gather some information about how much and what kind of memory are requested by the different benchmarks.

I used the information provided by the malloc_stats_file CDGC option, and generated some stats.

The analysis is done on the allocations requested by the program (calls to gc_malloc()) and contrasting that with the real memory allocated by the GC. Note that only the GC heap memory (that is, memory dedicated to the program, which the GC scans in the collections) is counted (internal GC memory used for bookkeeping is not).

Also note that in this post I generally refer to object meaning a block of memory, it doesn't mean they are actually instance of a class or anything. Finally bear in mind that all the figures shown here are the sum of all the allocations done in the life of a program. If the collected data says a program requested 1GiB of memory, that doesn't mean the program had a residency of 1GiB, the program could had a working set of a few KiB and recycled memory like hell.

When analyzing the real memory allocated by the GC, there are two modes being analyzed, one is the classic conservative mode and the other is the precise mode (as it is in the original patch, storing the type information pointer at the end of the blocks). So the idea here is to measure two major things:

• The amount of memory wasted by the GC because of how it arranges memory as fixed-size blocks (bins) and large objects that uses whole pages.
• The extra amount of memory wasted by the GC when using precise mode because it stores the type information pointer at the end of the blocks.

I've selected a few representative benchmarks. Here are the results:

bh allocation pattern

This is a translation by Leonardo Maffi from the Olden Benchmark that does a Barnes–Hut simulation. The program is CPU intensive an does a lot of allocation of about 5 different small objects.

Here is a graphic summary of the allocation requests and real allocated memory for a run with -b 4000:

We can easily see here how the space wasted by the GC memory organization is significant (about 15% wasted), and how the type information pointer is adding an even more significant overhead (about 36% of the memory is wasted). This means that this program will be 15% more subject to false pointers (and will have to scan some extra memory too, but fortunately the majority of the memory doesn't need to be scanned) than it should in conservative mode and that the precise mode makes things 25% worse.

You can also see how the extra overhead in the precise mode is because some objects that should fit in a 16 bin now need a 32 bytes bin to hold the extra pointer. See how there were no waste at all in the conservative mode for objects that should fit a 16 bytes bin. 117MiB are wasted because of that.

Here is a more detailed (but textual) summary of the memory requested and allocated:

Requested

Total

15,432,462 objecs, 317,236,335 bytes [302.54MiB]

Scanned

7,757,429 (50.27%) objecs, 125,360,510 bytes [119.55MiB] (39.52%)

Not scanned

7,675,033 (49.73%) objecs, 191,875,825 bytes [182.99MiB] (60.48%)

Different object sizes

8

Objects requested with a bin size of:

16 bytes

7,675,064 (49.73%) objects, 122,801,024 bytes [117.11MiB] (38.71%)

32 bytes

7,734,214 (50.12%, 99.85% cumulative) objects, 193,609,617 bytes [184.64MiB] (61.03%, 99.74% cumulative)

64 bytes

23,181 (0.15%, 100% cumulative) objects, 824,988 bytes [805.65KiB] (0.26%, 100% cumulative)

256 bytes

2 (0%, 100% cumulative) objects, 370 bytes (0%, 100% cumulative)

512 bytes

1 (0%, 100% cumulative) objects, 336 bytes (0%, 100% cumulative)

Allocated

Conservative mode

Total allocated

371,780,480 bytes [354.56MiB]

Total wasted

54,544,145 bytes [52.02MiB], 14.67%

Wasted due to objects that should use a bin of

16 bytes

0 bytes (0%)

32 bytes

53,885,231 bytes [51.39MiB] (98.79%, 98.79% cumulative)

64 bytes

658,596 bytes [643.16KiB] (1.21%, 100% cumulative)

256 bytes

142 bytes (0%, 100% cumulative)

512 bytes

176 bytes (0%, 100% cumulative)

Precise mode

Total allocated

495,195,296 bytes [472.26MiB]

Total wasted

177,958,961 bytes [169.71MiB], 35.94%

Wasted due to objects that should use a bin of

16 bytes

122,801,024 bytes [117.11MiB] (69.01%)

32 bytes

54,499,023 bytes [51.97MiB] (30.62%, 99.63% cumulative)

64 bytes

658,596 bytes [643.16KiB] (0.37%, 100% cumulative)

256 bytes

142 bytes (0%, 100% cumulative)

512 bytes

176 bytes (0%, 100% cumulative)

bigarr allocation pattern

This is a extremely simple program that just allocate a big array of small-medium objects (all of the same size) I found in the D NG.

Here is the graphic summary:

The only interesting part of this test is how many space is wasted because of the memory organization, which in this case goes up to 30% for the conservative mode (and have no change for the precise mode).

Here is the detailed summary:

Requested

Total

12,000,305 objecs, 1,104,160,974 bytes [1.03GiB]

Scanned

12,000,305 (100%) objecs, 1,104,160,974 bytes [1.03GiB] (100%)

Not scanned

0 (0%) objecs, 0 bytes (0%)

Different object sizes

5

Objects requested with a bin size of

128 bytes

12,000,000 (100%, 100% cumulative) objects, 1,056,000,000 bytes [1007.08MiB] (95.64%, 95.64% cumulative)

256 bytes

2 (0%, 100% cumulative) objects, 322 bytes (0%, 95.64% cumulative)

512 bytes

1 (0%, 100% cumulative) objects, 336 bytes (0%, 95.64% cumulative)

more than a page

302 (0%) objects, 48,160,316 bytes [45.93MiB] (4.36%)

Allocated

Conservative mode

Total allocated

1,584,242,808 bytes [1.48GiB]

Total wasted

480,081,834 bytes [457.84MiB], 30.3%

Wasted due to objects that should use a bin of

128 bytes

480,000,000 bytes [457.76MiB] (99.98%, 99.98% cumulative)

256 bytes

190 bytes (0%, 99.98% cumulative)

512 bytes

176 bytes (0%, 99.98% cumulative)

more than a page

81,468 bytes [79.56KiB] (0.02%)

Precise mode

Total allocated

1,584,242,808 bytes [1.48GiB]

Total wasted

480,081,834 bytes [457.84MiB], 30.3%

Wasted due to objects that should use a bin of:

128 bytes

480,000,000 bytes [457.76MiB] (99.98%, 99.98% cumulative)

256 bytes

190 bytes (0%, 99.98% cumulative)

512 bytes

176 bytes (0%, 99.98% cumulative)

more than a page

81,468 bytes [79.56KiB] (0.02%)

mcore allocation pattern

This is program that test the contention produced by the GC when appending to (thread-specific) arrays in several threads concurrently (again, found at the D NG). For this analysis the concurrency doesn't play any role though, is just a program that do a lot of appending to a few arrays.

Here are the graphic results:

This is the most boring of the examples, as everything works as expected =)

You can clearly see how the arrays grow, passing through each bin size and finally becoming big objects which take most of the allocated space. Almost nothing need to be scanned (they are int arrays), and practically there is no waste. That's a good decision by the array allocation algorithm, which seems to exploit the bin sizes to the maximum. Since almost all the data is doesn't need to be scanned, there is no need to store the type information pointers, so there is no waste either for the precise mode (the story would be totally different if the arrays were of objects that should be scanned, as probably each array allocation would waste about 50% of the memory to store the type information pointer).

Here is the detailed summary:

Requested

Total requested

367 objecs, 320,666,378 bytes [305.81MiB]

Scanned

8 (2.18%) objecs, 2,019 bytes [1.97KiB] (0%)

Not scanned

359 (97.82%) objecs, 320,664,359 bytes [305.81MiB] (100%)

Different object sizes

278

Objects requested with a bin size of

16 bytes

4 (1.09%) objects, 20 bytes (0%)

32 bytes

5 (1.36%, 2.45% cumulative) objects, 85 bytes (0%, 0% cumulative)

64 bytes

4 (1.09%, 3.54% cumulative) objects, 132 bytes (0%, 0% cumulative)

128 bytes

4 (1.09%, 4.63% cumulative) objects, 260 bytes (0%, 0% cumulative)

256 bytes

6 (1.63%, 6.27% cumulative) objects, 838 bytes (0%, 0% cumulative)

512 bytes

9 (2.45%, 8.72% cumulative) objects, 2,708 bytes [2.64KiB] (0%, 0% cumulative)

1024 bytes

4 (1.09%, 9.81% cumulative) objects, 2,052 bytes [2KiB] (0%, 0% cumulative)

2048 bytes

4 (1.09%, 10.9% cumulative) objects, 4,100 bytes [4KiB] (0%, 0% cumulative)

4096 bytes

4 (1.09%, 11.99% cumulative) objects, 8,196 bytes [8KiB] (0%, 0.01% cumulative)

more than a page

323 (88.01%) objects, 320,647,987 bytes [305.79MiB] (99.99%)

Allocated

Conservative mode

Total allocated

321,319,494 bytes [306.43MiB]

Total wasted

653,116 bytes [637.81KiB], 0.2%

Wasted due to objects that should use a bin of

16 bytes

44 bytes (0.01%)

32 bytes

75 bytes (0.01%, 0.02% cumulative)

64 bytes

124 bytes (0.02%, 0.04% cumulative)

128 bytes

252 bytes (0.04%, 0.08% cumulative)

256 bytes

698 bytes (0.11%, 0.18% cumulative)

512 bytes

1,900 bytes [1.86KiB] (0.29%, 0.47% cumulative)

1024 bytes

2,044 bytes [2KiB] (0.31%, 0.79% cumulative)

2048 bytes

4,092 bytes [4KiB] (0.63%, 1.41% cumulative)

4096 bytes

8,188 bytes [8KiB] (1.25%, 2.67% cumulative)

more than a page

635,699 bytes [620.8KiB] (97.33%)

Precise mode

Total allocated

321,319,494 bytes [306.43MiB]

Total wasted

653,116 bytes [637.81KiB], 0.2%

Wasted due to objects that should use a bin of

16 bytes

44 bytes (0.01%)

32 bytes

75 bytes (0.01%, 0.02% cumulative)

64 bytes

124 bytes (0.02%, 0.04% cumulative)

128 bytes

252 bytes (0.04%, 0.08% cumulative)

256 bytes

698 bytes (0.11%, 0.18% cumulative)

512 bytes

1,900 bytes [1.86KiB] (0.29%, 0.47% cumulative)

1024 bytes

2,044 bytes [2KiB] (0.31%, 0.79% cumulative)

2048 bytes

4,092 bytes [4KiB] (0.63%, 1.41% cumulative)

4096 bytes

8,188 bytes [8KiB] (1.25%, 2.67% cumulative)

more than a page

635,699 bytes [620.8KiB] (97.33%)

voronoi allocation pattern

This is one of my favourites, because is always problematic. It "computes the voronoi diagram of a set of points recursively on the tree" and is also taken from the Olden Benchmark and translated by Leonardo Maffi to D.

Here are the graphic results for a run with -n 30000:

This have a little from all the previous examples. Practically all the heap should be scanned (as in bigarr), it wastes a considerably portion of the heap because of the fixed-size blocks (as all but mcore), it wastes just a very little more because of type information (as all but bh) but that waste comes from objects that should fit in a 16 bytes bin but is stored in a 32 bytes bin instead (as in bh).

Maybe that's why it's problematic, it touches a little mostly all the GC flaws.

Here is the detailed summary:

Requested

Total requested

1,309,638 objecs, 33,772,881 bytes [32.21MiB]

Scanned

1,309,636 (100%) objecs, 33,772,849 bytes [32.21MiB] (100%)

Not scanned

2 (0%) objecs, 32 bytes (0%)

Different object sizes

6

Objects requested with a bin size of

16 bytes

49,152 (3.75%) objects, 786,432 bytes [768KiB] (2.33%)

32 bytes

1,227,715 (93.74%, 97.5% cumulative) objects, 31,675,047 bytes [30.21MiB] (93.79%, 96.12% cumulative)

64 bytes

32,768 (2.5%, 100% cumulative) objects, 1,310,720 bytes [1.25MiB] (3.88%, 100% cumulative)

256 bytes

2 (0%, 100% cumulative) objects, 346 bytes (0%, 100% cumulative)

512 bytes

1 (0%, 100% cumulative) objects, 336 bytes (0%, 100% cumulative)

Allocated

Conservative mode

Total allocated

42,171,488 bytes [40.22MiB]

Total wasted

8,398,607 bytes [8.01MiB], 19.92%

Wasted due to objects that should use a bin of

16 bytes

0 bytes (0%)

32 bytes

7,611,833 bytes [7.26MiB] (90.63%, 90.63% cumulative)

64 bytes

786,432 bytes [768KiB] (9.36%, 100% cumulative)

256 bytes

166 bytes (0%, 100% cumulative)

512 bytes

176 bytes (0%, 100% cumulative)

Precise mode

Total allocated

42,957,888 bytes [40.97MiB]

Total wasted

9,185,007 bytes [8.76MiB], 21.38%

Wasted due to objects that should use a bin of

16 bytes

786,400 bytes [767.97KiB] (8.56%)

32 bytes

7,611,833 bytes [7.26MiB] (82.87%, 91.43% cumulative)

64 bytes

786,432 bytes [768KiB] (8.56%, 100% cumulative)

256 bytes

166 bytes (0%, 100% cumulative)

512 bytes

176 bytes (0%, 100% cumulative)

Dil allocation pattern

Finally, this is by far my favourite, the only real-life program, and the most colorful example (literally =).

Dil is a D compiler, and as such, it works a lot with strings, a lot of big chunks of memory, a lot of small objects, it has it all! String manipulation stress the GC a lot, because it uses objects (blocks) of all possible sizes ever, specially extremely small objects (less than 8 bytes, even a lot of blocks of just one byte!).

Here are the results of a run of Dil to generate Tango documentation (around 555 source files are processed):

Didn't I say it was colorful?

This is like the voronoi but taken to the extreme, it really have it all, it allocates all types of objects in significant quantities, it wastes a lot of memory (23%) and much more when used in precise mode (33%).

Here is the detailed summary:

Requested

Total

7,307,686 objecs, 322,411,081 bytes [307.48MiB]

Scanned

6,675,124 (91.34%) objecs, 227,950,157 bytes [217.39MiB] (70.7%)

Not scanned

632,562 (8.66%) objecs, 94,460,924 bytes [90.08MiB] (29.3%)

Different object sizes

6,307

Objects requested with a bin size of

16 bytes

2,476,688 (33.89%) objects, 15,693,576 bytes [14.97MiB] (4.87%)

32 bytes

3,731,864 (51.07%, 84.96% cumulative) objects, 91,914,815 bytes [87.66MiB] (28.51%, 33.38% cumulative)

64 bytes

911,016 (12.47%, 97.43% cumulative) objects, 41,918,888 bytes [39.98MiB] (13%, 46.38% cumulative)

128 bytes

108,713 (1.49%, 98.91% cumulative) objects, 8,797,572 bytes [8.39MiB] (2.73%, 49.11% cumulative)

256 bytes

37,900 (0.52%, 99.43% cumulative) objects, 6,354,323 bytes [6.06MiB] (1.97%, 51.08% cumulative)

512 bytes

22,878 (0.31%, 99.75% cumulative) objects, 7,653,461 bytes [7.3MiB] (2.37%, 53.45% cumulative)

1024 bytes

7,585 (0.1%, 99.85% cumulative) objects, 4,963,029 bytes [4.73MiB] (1.54%, 54.99% cumulative)

2048 bytes

3,985 (0.05%, 99.9% cumulative) objects, 5,451,493 bytes [5.2MiB] (1.69%, 56.68% cumulative)

4096 bytes

2,271 (0.03%, 99.93% cumulative) objects, 6,228,433 bytes [5.94MiB] (1.93%, 58.61% cumulative)

more than a page

4,786 (0.07%) objects, 133,435,491 bytes [127.25MiB] (41.39%)

Allocated

Conservative mode

Total allocated

419,368,774 bytes [399.94MiB]

Total wasted

96,957,693 bytes [92.47MiB], 23.12%

Wasted due to objects that should use a bin of

16 bytes

23,933,432 bytes [22.82MiB] (24.68%)

32 bytes

27,504,833 bytes [26.23MiB] (28.37%, 53.05% cumulative)

64 bytes

16,386,136 bytes [15.63MiB] (16.9%, 69.95% cumulative)

128 bytes

5,117,692 bytes [4.88MiB] (5.28%, 75.23% cumulative)

256 bytes

3,348,077 bytes [3.19MiB] (3.45%, 78.68% cumulative)

512 bytes

4,060,075 bytes [3.87MiB] (4.19%, 82.87% cumulative)

1024 bytes

2,804,011 bytes [2.67MiB] (2.89%, 85.76% cumulative)

2048 bytes

2,709,787 bytes [2.58MiB] (2.79%, 88.56% cumulative)

4096 bytes

3,073,583 bytes [2.93MiB] (3.17%, 91.73% cumulative)

more than a page

8,020,067 bytes [7.65MiB] (8.27%)

Precise mode:

Total allocated

482,596,774 bytes [460.24MiB]

Total wasted

160,185,693 bytes [152.76MiB], 33.19%

Wasted due to objects that should use a bin of

16 bytes

26,820,824 bytes [25.58MiB] (16.74%)

32 bytes

85,742,913 bytes [81.77MiB] (53.53%, 70.27% cumulative)

64 bytes

18,070,872 bytes [17.23MiB] (11.28%, 81.55% cumulative)

128 bytes

5,221,884 bytes [4.98MiB] (3.26%, 84.81% cumulative)

256 bytes

3,400,557 bytes [3.24MiB] (2.12%, 86.93% cumulative)

512 bytes

4,125,611 bytes [3.93MiB] (2.58%, 89.51% cumulative)

1024 bytes

2,878,763 bytes [2.75MiB] (1.8%, 91.31% cumulative)

2048 bytes

2,760,987 bytes [2.63MiB] (1.72%, 93.03% cumulative)

4096 bytes

3,143,215 bytes [3MiB] (1.96%, 94.99% cumulative)

more than a page

8,020,067 bytes [7.65MiB] (5.01%)

Yaii! My beep will be back in the next kernel release :)

This is a note to let you know that I've just added the patch titled

    ALSA: hda - Add PC-beep whitelist for an Intel board

to the 2.6.35-stable tree which can be found at:
    http://www.kernel.org/git/?p=linux/kernel/git/stable/stable-queue.git;a=summary

The filename of the patch is:
    ../tmp/alsa-hda-add-pc-beep-whitelist-for-an-intel-board.patch
and it can be found in the queue-2.6.35 subdirectory.

If you, or anyone else, feels it should not be added to the stable tree,
please let <stable@kernel.org> know about it.


>From e096c8e6d5ed965f346d94befbbec2275dde3621 Mon Sep 17 00:00:00 2001
From: Takashi Iwai <tiwai@suse.de>
Date: Tue, 3 Aug 2010 17:20:35 +0200
Subject: ALSA: hda - Add PC-beep whitelist for an Intel board

From: Takashi Iwai <tiwai@suse.de>

commit e096c8e6d5ed965f346d94befbbec2275dde3621 upstream.

An Intel board needs a white-list entry to enable PC-beep.
Otherwise the driver misdetects (due to bogus BIOS info) and ignores
the PC-beep on 2.6.35.

Reported-and-tested-by: Leandro Lucarella <luca@llucax.com>
Signed-off-by: Takashi Iwai <tiwai@suse.de>
Signed-off-by: Greg Kroah-Hartman <gregkh@suse.de>

---
 sound/pci/hda/patch_realtek.c |    1 +
 1 file changed, 1 insertion(+)

--- a/sound/pci/hda/patch_realtek.c
+++ b/sound/pci/hda/patch_realtek.c
@@ -5183,6 +5183,7 @@ static void fillup_priv_adc_nids(struct

 static struct snd_pci_quirk beep_white_list[] = {
        SND_PCI_QUIRK(0x1043, 0x829f, "ASUS", 1),
+       SND_PCI_QUIRK(0x8086, 0xd613, "Intel", 1),
        {}
 };



Patches currently in stable-queue which might be from tiwai@suse.de are

queue-2.6.35/alsa-hda-add-pc-beep-whitelist-for-an-intel-board.patch

If you feel it should not be added to the stable tree, and let <stable@kernel.org> know about it, I kill you!.

Don't you love GMANE? I know I do, and now I can read my favourite RSS feeds via GWENE too. I even noticed it existence because I way a bunch of new newsgroups under the top-level name gwene.

There is a lot of activity lately in the GDC world, both the repository and the mailing list (mostly closed bug reports notification).

The reborn of GDC is a courtesy of Vincenzo Ampolo (AKA goshawk) and Michael P. (I'm sorry, I couldn't find his last name), but it received a big boost lately since Iain Buclaw (apparently from Ubuntu, judging from his e-mail address) got commit access to the repository in July.

They are working in updating both the DMD front-end and the GCC back-end and judging from the commits, the front-end is updated to DMD 1.062 and 2.020 (they are working on Druntime integration now) and the back-end works with GCC 3.4, 4.1, 4.2, 4.3 and 4.4 is in the works.

Really great news! Thanks for the good work!

Yes, I know I'm not Dijkstra, but I always wanted to do a considered harmful essay =P

And I'm talking about a very specific issue, so this will probably a boring reading for most people :)

This is about my research in D garbage collection, the CDGC, and related to a recent post and the precise heap scanning patch.

I've been playing with the patch for a couple of weeks now, and even when some of the tests in my benchmark became more stable, other tests had the inverse effect, and other tests even worsen their performance.

The extra work done by the patch should not be too significant compared with the work it avoids by no scanning things that are no pointers, so the performance, intuitively speaking, should be considerably increased for test that have a lot of false pointers and for the other tests, at least not be worse or less stable. But that was not what I observed.

I finally got to investigate this issue, and found out that when the precise version was clearly slower than the Tango basic collector, it was due to a difference in the number of collections triggered by the test. Sometimes a big difference, and sometimes with a lot of variation. The number usually never approached to the best value achieved by the basic collector.

For example, the voronoi test with N = 30_000, the best run with the basic collector triggered about 60 collections, varying up to 90, while the precise scanning triggered about 80, with very little variation. Even more, if I ran the tests using setarch to avoid heap addresses randomization (see the other post for details), the basic collector triggered about 65 collections always, while the precise collector still triggered 80, so there was something wrong with the precise scanning independently of the heap addresses.

So the suspicions I had about storing the type information pointer at the end of the block being the cause of the problem became even more suspicious. So I added an option to make the precise collector conservative. The collection algorithm, that changed between collectors, was untouched, the precise collector just don't store the type information when is configured in conservative mode, so it scans all the memory as if it didn't had type information. The results where almost the same as the basic collector, so the problem really was the space overhead of storing the type information in the same blocks the mutator stores it's data.

It looks like the probability of keeping blocks alive incorrectly because of false pointer, even when they came just from the static data and the stack (and other non-precise types, like unions) is increased significantly because of the larger blocks.

The I tried to strip the programs (all the test were using programs with debug info to ease the debugging when I brake the GC :), and the number of collections decreased considerably in average, and the variation between runs too. So it looks like that in the scanned static data are included the debug symbols or there is something else adding noise. This for both precise and conservative scanning, but the effect is worse with precise scanning. Running the programs without heap address randomization (setarch -R), usually decreases the number of collections and the variance too.

Finally, I used a very naïve (but easy) way of storing the type information pointers outside the GC scanned blocks, wasting 25% of space just to store this information (as explained in a comment to the bug report), but I insist, that overhead is outside the GC scanned blocks, unlike the small overhead imposed by storing a pointer at the end of that blocks. Even with such high memory overhead, the results were surprising, the voronoi number of collections doing precise scanning dropped to about 68 (very stable) and the total runtime was a little smaller than the best basic GC times, which made less collections (and were more unstable between runs).

Note that there are still several test that are worse for the CDGC (most notably Dil, the only real-life application :), there are plenty of changes between both collectors and I still didn't look for the causes.

I'll try to experiment with a better way of storing the type information pointers outside the GC blocks, probably using a hash table.

At last but not least, here are some figures (basic is the Tango basic collector, cdgc is the CDGC collector with the specified modifications):

Precise scanning patch doing conservative scanning (not storing the type information at all).

Precise scanning storing the type information at the end of the GC blocks.

Precise scanning storing the type information outside the GC blocks.

Here are the same tests, but with the binaries stripped:

Precise scanning patch doing conservative scanning (not storing the type information at all). Stripped.

Precise scanning storing the type information at the end of the GC blocks. Stripped.

Precise scanning storing the type information outside the GC blocks. Stripped.

Here are the same tests as above, but disabling Linux heap addresses randomization (setarch -R):

Precise scanning patch doing conservative scanning (not storing the type information at all). Stripped. No addresses randomization.

Precise scanning storing the type information at the end of the GC blocks. Stripped. No addresses randomization.

Precise scanning storing the type information outside the GC blocks. Stripped. No addresses randomization.

This is one of those threads where I want to drop my work on the GC and kill some of the D's main developers.

Please, at least learn a little about abort() before giving an opinion!

I've just published the git repository of my D GC implementation: CDGC. The name stands for Concurrent D Garbage Collector but right now you may call it Configurable D Garbage Collector, as there is no concurrency at all yet, but the GC is configurable via environment variables :)

It's based on the Tango (0.99.9) basic GC, there are only few changes at the moment, probably the bigger ones are:

• Runtime configurability using environment variables.
• Logging of malloc()s and collections to easily get stats about time and space consumed by the GC (option malloc_stats_file [str] and collect_stats_file [str]).
• Precise heap scanning based on the patches published in bug 3463 (option conservative [bool]).
• Runtime configurable debug features (option mem_stomp [bool] and sentinel [bool]).
• Other non user-visible cleanups.

The configuration is done via the D_GC_OPTS environment variable, and the format is:

D_GC_OPTS=opt1=value:opt2=value:bool_opt:opt3=value

Where opt1, opt2, opt3 and bool_opt are option names and value is their respective values. Boolean options can omit the value (which means true) or use a value of 0 or 1 to express false and true respectively. String options have no limitations, except they can't have the : char in their values and they have a maximum value length (255 at this moment).

At the moment is a little slower than the Tango basic GC, because the precise scanning is done very naively and a lot of calls to findPool() are done. This will change in the future.

There is a lot of work to be done (cleanup, optimization and the concurrent part :), but I'm making it public because maybe someone could want to adapt some of the ideas or follow the development.

See this small program:

template<typename T1>
struct A {
        template<typename T2>
        void foo_A() {}
};

template<typename T>
struct B : A<T> {
        void foo_B() {
                this->foo_A<int>(); // line 10
        }
};

int main() {
        B<int> b;
        b.foo_B();
        return 0;
}

You may think it should compile. Well, it doesn't:

g++ t.cpp -o t
t.cpp: In member function ‘void B<T>::foo_B()’:
t.cpp:10: error: expected primary-expression before ‘int’
t.cpp:10: error: expected ‘;’ before ‘int’

Today I've learned a new (horrible) feature of C++, foo_A is an ambiguous symbol for C++. I've seen the typename keyword being used to disambiguate types before (specially when using iterators) but never a template. Here is the code that works:

template<typename T1>
struct A {
        template<typename T2>
        void foo_A() {}
};

template<typename T>
struct B : A<T> {
        void foo_B() {
                this->template foo_A<int>();
                //    ^^^^^^^^
                // or: A<T>::template foo_A<int>();
                // but not simply: template foo_A<int>();
        }
};

int main() {
        B<int> b;
        b.foo_B();
        return 0;
}

Note how you have to help the compiler, explicitly saying yes, believe me, foo_A is a template because it has no clue. Also note that the template keyword is only needed when A, B and A::foo_A are all templates; remove the template<...> to any of them, and the original example will compile flawlessly, so this is a special special special case.

Yeah, really spooky!

In D things are more natural, because templates are not ambiguous (thanks to the odd symbol!(Type) syntax), you can just write:

class A(T1) {
        void foo_A(T2)() {}
}

class B(T) : A!(T) {
        void foo_B() {
                foo_A!(int)();
        }
}

void main() {
        B!(int) b;
        b.foo_B();
}

And all works as expected.

I'm sorry if I'm being a little repetitive about Futurama lately, but I just saw Proposition Infinity (6ACV04) which talks about egalitarian marriage between humans and robots (obviously parodying gay marriage legalization).

Well, it turns out, in case you didn't know, in Argentina, egalitarian marriage (AKA gay marriage) is legal since July 15th. This Futurama episode was aired on the July 8th, so it was kind of premonitory (leaving out the fact that the law in Argentina was in discussion long before that :).

Anyway, cheers to all homosexual couples that can get married now in Argentina, and to all robosexual couples that will be able to get married sometime after the year 3000 =P.

The attack of the killer app (6ACV03) is one of the best and geekest episodes of Futurama I've ever seen.

Making fun of iPhone and Twitter is really easy, and done everywhere now days, but Futurama does it in another level, in a level only geeks can enjoy that much. One clear example (leaving aside the title of the episode) is this little jewel:

Fry: Since when is the Internet about robbing people of their privacy?

Bender: August 6th, 1991.

Referencing the announcement of the creation of the WorldWideWeb.

I spent hours laughing at this Zuiikin' English videos.

I've done my regular aptitude full-upgrade and noticed the f*cking Adobe Flash player stopped working for amd64.

If you are experiencing the same, take a look at the workaround here.

Flash must die, DIE!

Today is a happy day. Even when the new Futurama season started in Comedy Central some weeks ago, I completely forgot about it and I didn't see any new episode until today.

I really missed the sensation of viewing a Futurama episode that I didn't see several times before (there were some movies, but it's not the same).

Oh! What a beautiful series! Really good first episode, very well spent 20 minutes.

How do I start describing this problem? Let's try to do it in chronological order...

The Problem

When done, I noticed a particular test that was notably slower in my implementation (it went from ~3 seconds to ~5 seconds). Here is the result (see the voronoi test, if you can read the labels, there is some overlapping because my effort to improve the graph was truncated by this issue :).

But I didn't recall it being that way when running the test manually. So I ran the test manually again, and it took ~3 seconds, not ~5. So I started to dig where the difference came from. You'll be surprised by my findings, the difference came from executing the tests inside the Makefile!

Yes, take a look at this (please note that I've removed all output from the voronoi program, the only change I've made):

$ /usr/bin/time -f%e ./voronoi -n 30000
3.10
$ echo 'all:' > Makefile
$ echo -e '\t$C' >> Makefile
$ make C="/usr/bin/time -f%e ./voronoi -n 30000"
/usr/bin/time -f%e ./voronoi -n 30000
5.11
$

This is not just one isolated run, I've tried hundreds of runs and the results are reproducible and stable.

From http://www.pixiesmusic.com/:

Pixies are Coming to Argentina!

Wednesday, October 6, 2010

Buenos Aires, Argentina

Luna Park

ticketing information to be announced shortly....

Whoa!

I've published my backup script (and named it Bacap, which is how it sounds when you pronounce "backup" in Spanish ;), as it suddenly became used by somebody else and received some patches =)

It has a simple home page for those wanting to see what is it about. I won't be doing formal releases thought, since is such a simple script, that would be overkill (just be thankful it has a git repo and a home page ;).

Sometimes performance issues matter more than you might think for a language. In this case I'm talking about the D programming language.

I'm trying to improve the GC, and I want to improve it not only in terms of performance, but in terms of code quality too. But I'm hitting some performance issues that prevent me to make the code better.

D support high level constructs, like delegates (aka closures). For example, to do a simple linear search I wanted to use this code:

T* find_if(bool delegate(ref T) predicate)
{
   for (size_t i = 0; i < this._size; i++)
      if (predicate(this._data[i]))
         return this._data + i;
   return null;
}
...
auto p = find_if((ref T t) { return t > 5; });

But in DMD, you don't get that predicate inlined (neither the find_if() call, for that matter), so you're basically screwed, suddenly you code is ~4x slower. Seriously, I'm not joking, using callgrind to profile the program (DMD's profiler doesn't work for me, I get a stack overflow for a recursive call when I try to use it), doing the call takes 4x more instructions, and in a real life example, using Dil to generate the Tango documentation, I get a 3.3x performance penalty for using this high-level construct.

I guess this is why D2's sort uses string mixins instead of delegates for this kind of things. The only lectures that I can find from this is delegates are failing in D, either because they have a bad syntax (compare sort(x, (ref X a, ref X b) { return a > b; }) with sort!"a < b"(x)) or because their performance sucks (mixins are inlined by definition, think of C macros). The language designer is telling you "don't use that feature".

Fortunately the later is only a DMD issue, LDC is able to inline those predicates (they have to inhibit the DMD front-end inlining to let LLVM do the dirty work, and it definitely does it better).

The problem is I can't use LDC because for some unknown reason it produces a non-working Dil executable, and Dil is the only real-life program I have to test and benchmark the GC.

I think this issue really hurts D, because if you can't write performance critical code using higher-level D constructs, you can't showcase your own language in the important parts.

The IT Crowd is back, a new season (4) has started yesterday.

Hurray!

Big news: DMD 64 bits support has just been started! It almost shadows the born of the new public DMD test suite (but I think the latter if far more important in the long term).

Solid new Album from The New Pornographers: Together. I will not say much more, if you now them, this new album won't disappoint you. If you don't know them, give them a try.

Probably my favourites are the catchy whistling of Crash Years, and the ballady My Shepherd.

Let me tell you my story.

I'm moving to a new jabber server, so I had to migrate my contacts. I have several jabber accounts, collected all over the years (I started using jabber a long time ago, around 2001 [1]; in that days ICQ interoperability was an issue =P), with a bunch of contacts each, so manual migration was out of the question.

First I thought "this is gonna get ugly" so I thought about using some XMPP Python library to do the work talking directly to the servers, but then I remember 2 key facts:

1. I use Psi, which likes XML a lot, and it has a roster cache in a file.
2. I use mcabber, which has a FIFO for injecting commands via the command line.

Having this two facts in mind, the migration was as easy as a less than 25 SLOC Python script, without any external dependencies (just Python stdlib):

import sys
import xml.etree.ElementTree as et

def ns(s):
        return '{http://psi-im.org/options}' + s

tree = et.parse(sys.argv[1])

accounts = tree.getroot()[0]

for account in accounts.getchildren():
        roster_cache = account.find(ns('roster-cache'))
        if roster_cache is None:
                continue
        for contact in roster_cache:
                name = contact.findtext(ns('name')).strip().encode('utf-8')
                jid = contact.findtext(ns('jid')).strip().encode('utf-8')
                print '/add', jid, name
                print '/roster search', jid
                g = contact.find(ns('groups')).findtext(ns('item'))
                if g is not None:
                        group = g.strip().encode('utf-8')
                        print '/move', group

Voilà!

Now all you have to do is know where your Psi accounts.xml file is (usually ~/.psi/profiles/<your_profile_name>/accounts.xml), and where your mcabber FIFO is (usually ~/.mcabber/mcabber.fifo, but maybe you have to configure mcabber first) and run:

python script.py /path/to/accounts.xml > /path/to/mcabber.fifo

You can omit the > /path/to/mcabber.fifo first if you have to take a peek at what mcabber commands will be executed, and if you are happy with the results run the full command to execute them.

The nice thing is it's very easy to customize if you have some notions of Python, for example, I didn't want to migrate one account; adding this line just below the for did the trick (the account is named Bad Account in the example):

if account.findtext(ns('name')).strip() == 'Bad Account':
        continue

Adding similar simple lines you can filter unwanted users, or groups, or whatever.

And all of this is thanks to:

• FLOSS
• Open file formats
• Unix philosophy

Thank god for that!

I've updated my Mutt Debian package with the NNTP patch to the latest Debian Mutt package.

If you have Debian testing/unstable and amd64 or i386 arch, just download and install the provided packages.

For other setups, here are the quick (copy&paste) instructions:

ver=1.5.20
deb_ver=$ver-8luca1
url=https://llucax.com.nyud.net/proj/mutt-nntp-debian/files/latest
wget $url/mutt_$deb_ver.dsc $url/mutt_$deb_ver.diff.gz \
   http://ftp.de.debian.org/debian/pool/main/m/mutt/mutt_$ver.orig.tar.gz
sudo apt-get build-dep mutt
dpkg-source -x mutt_$deb_ver.dsc
cd mutt-$ver
dpkg-buildpackage -rfakeroot
# install any missing packages reported by dpkg-buildpackage and try again
cd ..
sudo dpkg -i mutt_${deb_ver}_*.deb mutt-patched_${deb_ver}_*.deb

See the project page for more details.

By Peter Callesen.

Via IncredibleWorld.

This LWN large article looks like a very interesting read (specially for people like me that have a very vague idea about modern memory systems):

Ulrich Drepper recently approached us asking if we would be interested in publishing a lengthy document he had written on how memory and software interact. We did not have to look at the text for long to realize that it would be of interest to many LWN readers. Memory usage is often the determining factor in how software performs, but good information on how to avoid memory bottlenecks is hard to find. This series of articles should change that situation.

The original document prints out at over 100 pages. We will be splitting it into about seven segments, each run 1-2 weeks after its predecessor. Once the entire series is out, Ulrich will be releasing the full text.

The full paper in PDF format is also available.

After moving my home page (and this blog) from home to a proper (VPS) hosting, there were some problems because I have to migrate from Apache to LIGHTTPD.

The RSS feed was affected, so if you read this blog using an aggregator, you probably didn't received any updates in the last weeks. I also fixed some other problems that made the feed not W3C valid. Fortunately now it is. =)

For some unknown reason I had a new album by Cloud Cult (Feel Good Ghosts (Tea-Partying Through Tornadoes)) which I resisted to hear for about 2 years because I thought it was Coldcut! I didn't hear Coldcut very much either to be honest, because the little I heard I didn't like a lot.

Well, it turns out from time to time I like to hear my music collection in random order, and I was impressed by a song by Cloud Cult (and I still thought it was Coldcut until I started writing this post =P).

So I'm hearing the complete album and is plain genius, I'll have to check out their other albums soon.

The band have some very nice curiosities too. From Last.fm:

The band’s founder and singer Craig Minowa has a degree in Environmental Science, and his environmental, political, and social awareness is reflected in much of Cloud Cult’s music.

From Wikipedia:

In 1997, lead singer Craig Minowa formed Earthology Records on his organic farm, powered by geothermal energy and built partially from reclaimed wood and recycled plastic. This nonprofit label uses only recycled materials and donates all profits to environmental charities. The band also tours in a biodiesel van.

I think everyone know the Six Degrees of Kevin Bacon and The Oracle of Bacon. Well, now we have the same but for music connections!

Meet 6 Degrees of Black Sabbath.

It's easier to find paths with more than 6 degrees ;)

The CocoRosie sisters are back with a new album, Grey Oceans. After seeing their new video, I guess nobody would wonder why their music is labeled as freak folk.

Weird stuff! =P

It's been some time since I was surprised by a video. It reminds me of videos like Closer, almost any video from Tool, Teardrop, Black Hole Sun and the later Peacebone. One wonders where these people get their imagination (besides hard drugs, of course).

tomas Lindquist Olsen is back!

And now LDC is updated to work with LLVM 2.7. I hope this is not an isolated event and LDC development becomes as active as it used to be =)

The sources has been moved to Bitbucket too, so if you usually keep up to date you'll have to change the repo URL.

La dama y la muerte

After a couple of days of downtime, TPB sais:

OTOH, one of the creators launched, a couple of months ago, Flattr (from Wikipedia):

Flattr is a project started by Peter Sunde and Linus Olsson. Users will be able to pay a small monthly amount and then click buttons on sites to share out the money they paid in among those sites, sort of like an Internet tip jar. The minimum users will have to pay is 2 euros. Sunde said, "the money you pay each month will be spread evenly among the buttons you click in a month. We want to encourage people to share money as well as content."

In the beginning of the service Flattr itself will take a 10% of all the users monthly flatrate. It's currently in a closed beta but users can sign up for a beta invite code on their own site.

Let's see how it goes...

Description stolen from Wikipedia:

Romanzo Criminale (Crime Novel) is an Italian-language film released in 2005, directed by Michele Placido, a criminal drama, it was highly acclaimed and won 15 awards. It is based on Giancarlo De Cataldo's 2002 novel, which is in turn inspired by the Banda della Magliana true story. The Magliana gang was one of the most powerful Italian criminal associations, dominating Rome's drug, gambling and other kinds of crime activities from the early 1970s to 1992 (death of Enrico De Pedis). The gang's affiliates start their career kidnapping rich people, drug dealing (hashish, cocaine, heroin, etc.) from the 70s they started working with the Italian secret service, fascists, terrorists, the Sicilian Cosa Nostra, Camorra and many more. Some gang members are still alive, as inmates of an Italian prison, or justice collaborators (Massimo Carminati, Maurizio Abbatino).

Highly recommended.

It turns out GDB 7.0+ can be extended through Python scripts, for instance, to add pretty-printers. And it turns out GCC 4.5 comes with some good pretty-printers for GDB.

Do you want to see the result of that combination?

$ cat -n p.cpp
     1
     2  #include <string>
     3  #include <vector>
     4  #include <map>
     5
     6  int main()
     7  {
     8          std::string s = "hello world";
     9          std::vector<std::string> v;
    10          v.push_back(s);
    11          v.push_back("nice");
    12          std::map<std::string, std::vector<std::string> > m;
    13          m[s] = v;
    14          v.push_back("yeah");
    15          m["lala"] = v;
    16          return 1;
    17  }
    18
$ g++ -g -o p p.cpp
$ gdb -q ./p
(gdb) break 16
Breakpoint 1 at 0x400f86: file p.cpp, line 16.
(gdb) run
Starting program: /tmp/p

Breakpoint 1, main () at p.cpp:16
16              return 1;
(gdb) print m
$1 = std::map with 2 elements = {
  ["hello world"] = std::vector of length 2, capacity 2 = {"hello world", "nice"},
  ["lala"] = std::vector of length 3, capacity 3 = {"hello world", "nice", "yeah"}
}
(gdb)

Nice, ugh?

The only missing step is configuration, because most distribution don't do the integration themselves yet (or don't have packages with the scripts).

Here are 3 quick steps to make it all work:

$ mkdir ~/.gdb # can be stored anywhere really
$ svn co svn://gcc.gnu.org/svn/gcc/trunk/libstdc++-v3/python ~/.gdb/python
$ cat << EOT > ~/.gdbinit
python
import sys
sys.path.insert(0, '/home/$HOME/.gdb/python')
from libstdcxx.v6.printers import register_libstdcxx_printers
register_libstdcxx_printers (None)
end
EOT

That's it!

If like to suffer once in a while you can get the raw values using /r:

(gdb) print /r m
$2 = {_M_t = {
    _M_impl = {<std::allocator<std::_Rb_tree_node<std::pair<std::basic_string<char, std::char_traits<char>,
std::allocator<char> > const, std::vector<std::basic_string<char, std::char_traits<char>, std::allocator<char> >,
std::allocator<std::basic_string<char, std::char_traits<char>, std::allocator<char> > > > > > >> =
{<__gnu_cxx::new_allocator<std::_Rb_tree_node<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char>
> const, std::vector<std::basic_string<char, std::char_traits<char>, std::allocator<char> >,
std::allocator<std::basic_string<char, std::char_traits<char>, std::allocator<char> > > > > > >> = {<No data fields>}, <No
data fields>},
      _M_key_compare = {<std::binary_function<std::basic_string<char, std::char_traits<char>, std::allocator<char> >,
std::basic_string<char, std::char_traits<char>, std::allocator<char> >, bool>> = {<No data fields>}, <No data fields>},
_M_header = {
        _M_color = std::_S_red, _M_parent = 0x6070b0, _M_left = 0x6070b0,
        _M_right = 0x607190}, _M_node_count = 2}}}

Looks more familiar? I guess you won't miss it! =P

I accidentaly removed a couple of files from a FAT partition, and even when undeleting files from a FAT patition is possible from ancient times, it took me some time to find a tool that worked in Linux and was packaged.

So in case you find yourself with this very same need, search no more: meet TestDisk.

I like my flickr, but I reached the limit of 200 photos. I have a few options:

1. Pay USD25/year for a pro account.
2. Look for another image hosting.
3. Don't care about old photos not being shown.
4. Start hosting the photos myself.

Since I'm starting to pay a hosting for my home page (and this blog), I'll go with option 4 :)

I'll use this same blog for that, but I will use the pic tag. I've created a new feed in feedburner, so you can use directly that one. The photos will not have a language tag (they will probably be always in english), this way the other feeds (en, es and D) will not get polluted by photos; so if you're reading this post trough one of those and you want to see the photos I'll post, you probably want to switch to the general feed or subscribe to the new feed too.

Failed... Twice!