(Click to view the bigger version.)

Unfortunately, there’s way too much truth behind it.

CAN is a bus system that has some pretty neat features. It can handle a lot of devices, can achieve pretty good bit rates, it has very good error handling and bus arbitration functions, and so on. It was originally designed for use in vehicles (and in fact, any car manufactured in the last 10 or 20 years or so is largely built around tons of individual devices talking on a CAN bus) but of course there’s a lot of other situations in which the features offered by CAN are useful.

I’ve been trying out the new MPLAB X (which is still in beta). There are some major shortcomings, but overall it’s starting to look like a pretty good IDE, which is nice because I absolutely hate the old MPLAB. Compiling is done using Hi-Tech C.

Now, if you were to ask my opinion on Hi-Tech C a week or two ago, you’d get a pretty negative review. I’ve been having issues with it. Weird issues, like global variables not working (no matter what you write, they always read as 0 unless you access them via a pointer, which is a bit of a stupid and inefficient workaround). I’ve also been running it in Lite mode, rather than Pro, because in Pro mode it uses the big Omniscient Code Generation – an optimization technique which is actually quite good, except that it always seemed to turn out horribly bad code that performed nothing like it was supposed to.

It turned out that all of those issues were caused by the extended instruction set being enabled in the PIC’s configuration bits. What happens is that that bit doesn’t only make a few extra instructions available (8, in fact) that are primarily of interest to a C compiler, bit it also changes the way memory addressing is accomplished. Thus, if the extended instruction set is enabled while the compiler isn’t generating code that uses those extended instructions (or specifically, its way of addressing memory), things go wonky. Big time. Turn the extendend instruction set off, and everything works like a charm (OCG included, cutting the code size in half!). Today, I’m pretty darn happy with Hi-Tech C.

Of course, the user manual doesn’t make a single reference to the extended instruction set (I’d say this is exactly the sort of thing that should be mentioned in the manual, but yeah. It’s not).

Also, for the CAN part, I’ve been using a setup with the MCP2515 CAN controller IC, paired with an MCP2551 transceiver. By my initial look at the datasheets, this should have been easy, until the moment I got to actually using it and had to figure out how the bit timing configuration works (and it would seem that I havn’t been the only person to struggle with this). As it turns out, once you grasp it it’s simple enough, and since I havn’t been able to find a functional calculator tool on the Interwebz (just one for the older MCP2510, which isn’t entirely compatible), I might just find some time to write such a calculator.

Still, with the bit timing issue aside, the packets that one of two CAN nodes is transmitting don’t seem to be making it to the other end, for some reason. That’s a thing to figure out once I have the opportunity to put an oscilloscope in the mix and see what’s going on.

Waaaay back, when everyone was using 28k8 dial-up modems, in the time of bulletin boards and Gopher (which is kind of the predecessor of the World Wide Web), things were simple. Granted, the number of things you could do with them was limited as well, and back in that time computers (to most people) were arcane and mystical devices that only superhuman-like beings could properly operate. But if you happened to be one of those who could, things were simple. The Gopher protocol was a simple protocol.

But then, computers slowly became affordable. And at some point, broadband internet (or more accurately described, non-dialup) became affordable too. And people started using the Internet. They were the times of early Internet Explorer and Netscape (the Dutch translation of which, at the time, probably due to not very knowledgeable translators, translated “cookies” – as in, the bits of information stored on your computer by a website – were translated as “koekjes” – the baked dough that we have all come to love. I am still disappointed that enabling the option “koekjes altijd accepteren” – or “always accept cookies” – never delivered me any physical yumminess.). The Web slowly developed. There came extensions to HTML. Websites had frames – one of those things that probably seemed like a good idea at the time, but turned out to be really stupid – and at some point dynamic websites became commonplace. Yahoo! GeoCities was full of animated GIFs, marquees, and blinking text. CSS and JavaScript and Flash and ActiveX and all sorts of other things popped up all over the place.

Today, we are feeling the result of the rapid but uncontrolled development that the Web went through. There are a bunch of different browsers, and none of them interpret HTML exactly the way they should. Instead of finally building a definitive standard, HTML5 is attempting to accomodate every broken webpage that was ever made and now they have decided that it won’t really have version numbers, let’s instead make it an ‘adaptive standard’ and change it as we go, just to make things easier on web developers (and browser builders). Every browser does JavaScript a different way, and writing a script that works accross all browsers is a tedious process involving many dirty hacks to implement a gazillion different methods to do the same task, depending on which browser it’s running in. (And yes, there are frameworks and things that try to make it easier, but that’s just throwing another layer of crap on top of an already broken system. Issues should be resolved at their origin, not by trying to painstakingly work around them.)

On the modern Web, if you want to make yourself as a web developer, there is just so much you need to go through. Not even to make a website that is impressive, no, even something that lives up to the user’s expectactions of an average website is a monstrous task. The basic markup language, HTML, is not too difficult too grasp. CSS makes it easier to create a consistent look and feel over a number of pages (really can’t do without it), and JavaScript allows us to do all sorts of interactive client-side things (once you get it to work, that is). But the websites must also be dynamic, so we bring in PHP, MySQL – suddenly we need to be familiar with programming and databases, both of which are huge subjects to master – and if you’re going to be anywhere near fancy, you’ll need to do at least a bit of AJAX. And if learning how to deal with all of those things isn’t bad enough on its own, there are a ton of other things to worry about: search engine optimization, statistics tracking, advertising, usability, accessability, and so on.

Oh, and everything should have a very sleek and nice look. You should become an expert at Photoshop (or whatever graphics design program you prefer) as well.

As great and useful as the World Wide Web may be, to me, web development feels like one half-broken technology on top of the other, where the developer has to go to greater lengths than ever to come up with a decent result. And it’s not looking like there will be improvement anytime soon.

Frankly, I am tired of it. There has been a time when I liked doing web development, but that’s gone.

Can we please put all the crap aside and go back to when things were simple? Or at least start over from scratch, and this time, we’ll do it right. Properly define stuff and account for every possibility that we may need to deal with, and have a simple way to add what we need if we forgot to add it from the start. And we’ll have browsers implement everything in the same way (or just not re-invent the wheel, and stick with a single implementation of the core engine and use that accross all browsers/platforms). And the world will be a much nicer place because of it.

Until then, I’d like my 28k8 back, please.

The only real issue I was having is getting it to work on my laptop, which runs Windows 7, 64-bit. The device uses a standard FT2232L chip, but with a customized Vendor/Product ID, so the drivers from the Olimex website won’t work with it directly, and as you may know Windows 7 can be a real bitch with drivers, especially in 64-bit.

Anyway, I finally got it to work and figured I’m not the only one who ran into this problem, so I thought I’d share the driver I used. (I modified it from the standard FTDI drivers).

To install, unzip the folder somewhere, go to the Device Manager, open the properties for the ARM-USB-OCD (it shows up twice: one for the JTAG interface, one for the serial port), find the ‘Update Drivers’ button, browse to the folder where you extracted the drivers, and voila. It will probably ask you to confirm you want to install an unverified driver, obviously you’ll have to accept there. No worries, it’s just the FTDI driver modified to accept the ARM-USB-OCD’s VID/PID combination.

Anyway, here it is: Olimex ARM-USB-OCD drivers for Windows 7 64-bit (and pretty much all other Windows versions too) – 1.18 MB

A couple of years ago I used Lua quite a bit (only writing Lua scripts, not applications that employ Lua). Back then I used to know it pretty well, but that was years ago. Not using the knowledge for so long combined with updates to the language make it necessary to re-learn a lot of things. But fortunately, Lua is actually a very simple language, it doesn’t come with tons of things we don’t need, the syntax is easy, and it has a couple of nice features.

One thing I love about Lua is the ability for functions to return more than 1 value. In pretty much any other language, a function can only ever return a single value, but often we want to return more data than that. In C/C++ the “obvious” workaround is to use additional function arguments, which will be pointers to the location where the result data is written to. Lua, on the other hand, does allow functions to return more than one value.

So instead of a complex function call with lots of arguments where you’re not really sure if they actually affect the behaviour of the function or if they are used to pass a result back, we can do this in Lua:

result1, result2, result3 = somefunction( argument1, argument2, argument3 )

Getting the basics of Lua working on your system isn’t that hard. It is downloaded in source form from the website, and needs to be built into a library (which you’ll also use if you write any applications that use Lua), the Lua interpreter (lua.exe), and the Lua compiler (luac.exe), which pre-compiles Lua scripts into bytecode (optional but does increase performance a bit). On Linux this is of course an extremely trivial task (tar xzvf lua-5.1.4.tar.gz, cd lua-5.1.4, make, make install), on Windows it’s a bit more work but still easy enough.

Then comes writing a little test application that calls an external Lua script, followed by making said test application a bit more complicated (by setting variables, adding functions, calling functions inside the Lua script, and things like that). To the untrained eye a C/C++ program that embeds Lua may seem complicated, but it’s really not that bad: Lua uses a stack for most of its work, and once you see how it works, it’s actually quite easy to work with. A lot of the work you do is just pushing or popping values to/from the stack.

I wonder how easy (or difficult) it will be to make Lua run from an embedded CPU.

Another interesting idea: Lua. As in, Lua, the scripting language that powers a lot of games, among other things. It would be interesting to run that from an ARM, again with the SD card for storage (its awesome if you can store your scripts somewhere). The libraries are small enough, if I recall correctly. I think it could work.

Something else I’ve tried out a few days ago is using memcached. Memcached is a small server program that runs in the background (as a service on Windows machines, or as a daemon on Linux) and uses spare RAM to store data. It can also run distributed, with a bunch of servers all throwing their spare memory into a big pool which can conveniently be accessed as if it was on a single machine. PHP has a standard plugin that can communicate with memcached – what can you do with that, you wonder? Well, as the name suggest, memcached is a cache which stores data (in key/value) pairs in memory. Basically your script can store data in it so you don’t have to fetch it from the database (saving you from executing a time-consuming query and putting load on the database server).

It is actually quite efficient – all of the world’s big websites (especially a lot of social networks and such) use it. Some sites even have massive amounts of servers dedicated to it. In such setups getting everything right is tricky (you want to be sure that everything is consistent – with a large distributed system it’s easy for something to go out of sync, potentially causing a lot of trouble), but it is quite useful.

I played with it for a bit and it turns out that memcached is really very easy to work with. Installation on my Windows box was a matter of putting the executable in a folder and running it command-line with a few options that make it install itself as a service, after which it is permanently available; accessing it from PHP takes no more than adding a standard plug-in to your php.ini file, restarting Apache, and calling the right functions. It’s actually very easy to work with. A quick measurement told me that, when loading some configuration data using memcache vs. loading it from the database, doing it from memcache is actually twice as fast – and that’s for a trivial task.

Unfortunately you’re not going to get memcached support with any basic shared hosting plans (and VPSes generally don’t have the spare memory capacity). It takes a bunch of setting up, securing, and firewalling to make it run securely in a live environment and you can’t let multiple users share the same memcached host without them being able to access eachother’s data. It’s a bit of a downside, but memcached was never intended to be used in a shared environment, so it’s not a big deal.

The project I’m working on right now is actually getting full-blown memcache support, so it will cache anything that can reasonably be cached (within configurable limits), making the whole thing run faster and using far less database queries.

Next up, the template engine. I’ve already written a template engine quite a while ago (several actually) – but I wanted to make some improvements and ended up rewriting most of it. If you don’t know, a template engine essentially separates presentation and program logic, which is very useful if you want to encrypt the actual PHP code without removing the ability to edit the HTML, or just generally to organize things – it makes it very easy to figure out where you need to be to change something, and even non-programmers can easily edit the layout of the page this way.

The previous incarnation of my engine supported variables and blocks. Variables are well, variables, and blocks can either be repeated any given number of times (the contents of variables can be varied for each instance – so this is what you would use to fill a table with data, for instance) or entirely hidden. This is sufficient in a lot of cases, but I wanted to add more: conditional statements, loops (other than repeated blocks), proper caching, and so on.

Previously, all parsing was done in the final processing function, which did everything by reading files and releasing a bunch of string manipulation functions on them. It worked, but the code was complicated as fuck and debugging (let alone adding something new) is a whole new kind of hell by itself. The new version uses a system not unlike what a compiler for, say, C, does: it splits the input data (the template file(s)) into a bunch of tokens (textual data, variables, blocks, etcetera), runs a lexical analysis check (grammar basically – make sure that the end if your if statement does not precede the actual if statement, for instance), and finally, goes over the list of tokens (using a specialized stack), combining it with the dynamic data (contents of variables, block instances, etcetera) to produce output.

That all sounds terribly complicated, and well, it is. One major advantage though is that now 90% of the processing effort (parsing and analyzing the template) can be stored for later use, and the engine does that either by files, or by sending the data to memcached (the latter being preferred, since it doesn’t involve hard drive activity, which is relatively slow). The result is a very powerful and fast system, in which it is actually possible to add a new feature, should the need for it arise.

The whole beast has a ton of features – variables (both local and global), blocks (plus ‘empty blocks’, something that gets shown if there are no instances of the block), conditional statements (if/elseif/else – admittedly expression handling is very basic at the moment, but that’s for future improvement), caching, variable modifiers, a bunch of debug features, tools like optionally stripping HTML comments from the final output – and the whole thing weighs in at ‘only’ 2650 lines of code (including tons of comments and whitespace) / 73.2 kilobytes (all in a single file). It’s definately not finished yet, but I’m quite satisfied already

Oh, and also, I now use NuSphere PhpED. Most of the time, IDE’s mostly manage to annoy the crap out of me, but this one actually works instead of being a constant pain in the ass. I’d definately recommend it.