Complex PHP daemons and Gearman

by on Mar.13, 2010, under internet, Linux, realsauce

Well I need to redo a property handling system from scratch. Code to move properties from one site to another.
Not only does it need to do that, but it also needs to only send what has changed, and it needs to do it very quickly.

So that means I’ll need a complex forking PHP daemon, one master to set tasks and then some slaves to focus on specific types of property transmission (e.g. REAXML) and then some more slaves for just minor tasks which can be run in parallel. E.g. fetching images.
All the communication will be via Gearman. Its a lot easier that way because Gearman handles queuing and can also do priorities.

Most methods of moving properties around is just XML deltas where a element is only specified if it has changed. When a new property is made then the full data is sent. There are hacks to emulate that, but I will be going will a full history system where every single change is stored, and if required can be replayed.

The main thing I’m not looking forward to doing is working on the code which makes sure that all the separate slaves are still running.
It needs to be able to reset its self if anything goes wrong, and log errors. Not easy stuff.
I’ll probably create some kind of heatbeat system.

Leave a Comment more...

NGW100 is running

by on Apr.29, 2008, under AVR32, Linux, NGW100

Well I got a replacement NGW100 to replace the one I borked.
That was a expensive mistake. :(

I’ve got it successfully modded so it can now be USB powered. :)
There is a convenient hole next to the USB port which is the USB’s 5v rail.
You just need to run a wire from there to the positive pin of the power supply, next to the double set of IO pins.

I put a diode in place as well so I can swap power supplies on the fly and 12v cant go and fry the USB port. ;)
I also put some heatshrink over it to make it look professional and reduce the chances of shorts.
Its a dead simple mod to make and its pretty useful.

I’ll be posting pics as soon as I can make my card reader work.

Now if only I could build a toolchain successfully. :(

2 Comments more...

The NGW100 is neat

by on Apr.01, 2008, under AVR32, Linux, NGW100

Well I got my NGW100 last night. :D

If you dont know what it is, its basically a tiny board which runs Linux.
It features two network ports, SD slot, serial pot and a slave USB port and its designed to be a simple yet powerful Network GateWay (NGW).

Unfortunately its already borked. :(
I was upgrading the ram chip on it from 32mb to 64mb and a drop of solder hit the Flash chip.
That chip just happens to have 0.5mm leads which is extremely tricky to solder correctly.

Fortunately I got a 0.8mm soldering iron tip today so tomorrow morning I should be able to recover it.

Pictures will come when its working. :P

Leave a Comment more...

Looking for something?

Use the form below to search the site:

Still not finding what you're looking for? Drop a comment on a post or contact us so we can take care of it!


A few highly recommended websites...