Work related posts have been moved.

9 November, 2010

My work and computing related posts are now at

If you have come here from a work-related perspective (computing, semweb, bioinformatics, math). Perhaps you could go there right now and not read the gory personal stuff here.

Use the wordpress rss mechanism to follow categories:

Follow Paul Murray on Quora


It’s a bad idea to shoot terrorists

13 November, 2018

You know, people who are afraid will do pretty much anything. Morality goes out the window. It’s kinda the nature of fear – it shuts off all those higher brain functions that make us civilised. That’s how torture works, you know. It’s not the pain: it’s the fear. That’s what causes wartime atrocities. That, and the predator/prey circuitry in the brain, which is simply the flip side of fear. Every demagogue knows it. It’s a permanent truth of human affairs. The only thing more effective is hunger. Or being smothered.

I have an insight into Islamic extremists that most Australians simply don’t have. I was once sincerely religious. I believed in a literal, actual Hell where God would send people to be burned alive forever. I truly did, I thought it was a real thing.

The fear of that place is the bedrock of religious faith. When everything else is stripped away, all the clever arguments, all the chop-logic, it’s that terrifying thought “but what if it’s all true?” that brings the believer back. When you truly, deeply believe in the reality of hell, the one thing you want more than anything else is simply to not be condemned to go there. It’s a background, an undertone to your whole life. Subliminal. It conditions all your choices. Nothing else really matters, in comparison.

Christianity is a bit different from Islam in a couple of respects. In Christianity, everyone deserves to go to hell. Everyone. Deserves. Because Adam ate the apple. Everyone should go to hell, forever. The situation is so absolute that God had to pull some legal sleight-of-hand to make it possible for people to not go there.

But in Islam, Allah works on sort of a points system. Allah is merciful, Allah is forgiving. You can skip a few prayers and make it up later, no problem. All Allah wants is that you be Islamic enough. The problem being, of course, that you can never really know, never really be sure if you have been Islamic enough for Allah.

So you have these young men, and they have lived a western life – not taking the faith all that seriously, drinking occasionally, not being careful to eat hallal, maybe having sex with western whores, it’s no problem: they’ll make it up later.

And then one day they have a crisis of conscience. Maybe they hear a sermon. Maybe they read the Koran. And it all comes crashing back. Oh shit: I’m going to go to hell.

Happily, there’s one thing that makes up for any amount of sin: being killed while committing jihad.

Those terrorists, they want to die. It’s the whole point of the exercise. They want to get shot by the cops. And they are not doing it to spread Islam, they are not really doing it for ISIS, they are not doing it because it’s right, or holy. They are doing it purely and entirely to get their personal ticket to heaven, to avoid the other place. It’s a deeply selfish act, but that’s the nature of fear. Real, bone-deep, religious fear. They don’t care about anything but avoiding the other place.

Killing them is a bad idea. Because it will not deter the next one: it will encourage them. Nonreligious, or conventionally religious people just don’t get it.

Shoot ’em in the leg. Tase ’em. Tackle ’em. Absolutely do not kill them. No matter how much they deserve it. Lock ’em up for life, and never belive a word they say about promising not to do it again. The only thing that will deter these private acts of jihad will be if jihad doesn’t work.

Some thoughts about Saudi Arabia

23 October, 2018

So, why does the USA cuddle up to Saudi Arabia, a nation whose state religion is a form of Islam that requires the death or forced conversion of everyone who isn’t a Muslim? How come eleven (?) Saudis, organised by a the son of a Saudi millionaire, fly a plane into the WTC and damn near manage to fly planes into the Pentagon and the White House, and Bush sends the army to attack – wait for it – Iraq? Why did Obama have to go to Saudi Arabia and kiss ass?

It’s about the oil, dummy. It always was. It really is that simple.

The oil under the sands of the Arabian Peninsula is the greatest material prize in human history. If we discover Lasseter’s Reef, all that will happen is that the price of gold will nosedive. Gold isn’t all that useful. But oil – oil is another story. Never mind the energy – where do you think plastics come from? Fertilizer? You think we want to go back to making things out of wood and sheetmetal? You think we want to return to an age when almost everyone was engaged in simply growing the food?

Back in the 1950’s, the USA purchased it. Made a trade for it. The oil. All the oil. Oh sure, they didn’t take delivery right away, but what they bought was a right to unlimited amounts of the stuff. The deal was: the Sauds would sell their oil only in US dollars (which the reserve bank of the USA prints out of thin air for nothing), and would invest their profits only in US treasuries (a debt that you never have to make good is no debt at all, and in any case US treasuries have been paying 0% for over a decade).

In return, the USA would provide the House of Saud with “security”. That’s the USA’s main export, you know. “Security”. The wars are trade shows. The military bases are shop-fronts. All selling “security”. And what might “security” be? It’s when someone agrees to kill your enemies for you. Oh, and also to not kill you. Call it “protection”, if you like.

That’s why “gas” in the USA is so cheap – they are always shocked at the price of petrol when they come overseas.

This isn’t some conspiracy theory, by the way. It’s well-established history. The US/Saudi agreement established the “petrodollar”, on which the whole economy of the late 20th century was based. You have heard the term, right? It is totally a thing.

But, there’s a couple of things going down at present. First, Saudi Arabia will run out of the stuff – God isn’t making any more at present. Oh, there’s still plenty left, but we can see the horizon. The day will come – maybe in our lifetimes – when SA can no longer uphold its end of the deal.

Second, the USA has been busy losing wars for several decades, now. Turns out that you can’t actually defeat a modern nation by anything short of genocide. The days are long gone when defeating a nation was a matter of forcing its king to bend the knee and job done. Genocide, you know, is an expensive affair when the people you are trying to genocide shoot back and plant IEDs (unlike German Jews, or Tasmanian Aboriginies). That is: it’s starting to look like the USA can’t hold up its end of the deal either. Gays and women in the military (something abhorrent to Islam), and Central Americans streaming over the undefended US border. It’s not a good look, you know?

Machiavelli said that the primary concern of the Prince is war. If you can’t warfight (as they put it these days), then you don’t have a principate. Or an empire, as the case may be.

The USA has unrivalled control of airspace. How much gas, do you think, does a fighter jet or a bomber consume? How much diesel does an aircraft carrier – let alone a carrier group – chew through every minute? How much does a tank, an APC, a chopper need? And how much do you need to run the generators that power the electronic frippery that no modern infantryman can fight without?

Oh sure, if the price of oil rose dramatically, the USA and the west could still afford to field an army and blow the shit out of places that choose to diss us. Or can we? Can we really? Money, money, money – it’s seriously a thing. “Gold is the sinews of war”. Machiavelli disagreed, but he may have been wrong about that one.

So Trump can pound the table about whathisface the journalist, but the USA cannot break its symbiosis with Saudi Arabia. Not without doing it the hard way (or inventing Mr Fusion – the power source of the future).

But neither can King Whatshisname turn off the spigot, or raise the price to $400 a barrel (same thing, really) as he has threatened to do. Because the West must have its oil, the military must have its oil, and if the Saudis attempt to withhold it in any serious way, we will just fucking come and take it. If the deal is off, then that means the deal is off, doesn’t it? Kinda important, when one of the terms of the deal is “protection” and you yourself are a family man.

Oh sure, these days it wouldn’t be as overt as that. There’ll be a “Color Revolution” or some shit. But that’s how it is, and everyone who pays attention knows it. Remember Saddam Hussein, who decided to try to sell his oil to the Europeans in Euros? Yeah. That.

This will all simmer down and be forgotten. So what if the Saudis decided to off some dissident, and a whole bunch of people are upset about it? Neither side can actually do anything about it, public outrage notwithstanding. Happily, people are fickle, stupid, and have short memories. It will go the way of all the other crises.

The world will just keep rolling on exactly like it is until the oil runs out, or the USA cannot field an effective fighting force, or someone actually does invent Mr Fusion.

Everyone who pays attention knows it.

The Honeybadger

6 October, 2018

So, The Bachelor. Why do I care? Because I don’t dare browse my usual internet content at my new job, I limit myself to because there’s a link to it in the SOE setup, and is loaded with crap about The Bachelor.

Seems The Honeybadger knocked back all the ladies. What a bastard! So many to pick from – he owed it to Australia to marry one of ’em!

Here’s what I think.

We can all agree, I’m sure, that it would be unreasonable to pick a girl and tell a dude he has to marry her. I’d say it would also be unreasonable to pick a dozen mostly identical girls and tell a dude to pick one. And all those bachelorette hopefuls were mostly identical.

“Identical?” you exclaim? “Identical? Why – some were blonde, and some weren’t!”

Were any of ’em – for instance – short? Maybe Mr Badge deep down yearns for a little pocket rocket to make his life complete. Were any of ’em – say – thicc? A little cushion for the pushin? Or: did any of ’em have truly big tits?

Nope, nope, and nope. They were all tall-ish, skinny models.

Were any of ’em asian? Maybe Mr Cummins has yellow fever. Maybe he has a thing for dusky temptresses from the subcontinent, so: were any of ’em Indian? How about black? Or – going the other way – were any one of these bachelorettes a milky-translucent-skinned green-eyed ranga? Molly Ringwald had her fans, you know.

But let’s get away from the merely physical.

Were any of ’em shy? Or demure? Or were all of them perfectly comfortable in a dress with a “neckline” plunging all the way to the navel?

Were any of ’em – oooh, I dunno – a fiery Mediterranean type? Maybe Mr Cummins has a thing for women who yell in Spanish or Italian, or Maltese women who truly believe in revenge.

Were any of ’em sincerely religious?

Naaah. There very format of the show makes it impossible. They were all models looking to get their faces and bodies on TV.

Were any of them really financially smart, a self-made millionairess from her own investments? Was any of them a waify, distracted art-girl? A nose-ringed anarchist? Or how about a country girl that can butcher a whole pig, no worries?

I’m sure at least some of them wanted children. So: how many of them could cook a family meal? Do a roast like it’s not really a big deal, it’s just what we usually have on a Sunday?

See what I’m saying? Don’t get me wrong: don’t accuse me of saying that a girl must be all of these things. Being one of them, or one of the millions of other types of individual is perfectly fine. There’s nothing in the least wrong with being a skinny cookie-cutter model with a white girl ass and no hips and I’m sure all of these women have their admirers. But insisting that Nick is obliged to marry a girl like that? When that’s just not what he happens to be into?

Hell no.

At least he takes marriage somewhat seriously. Seriously enough, at least, to not commit to someone because a TV producer tells him that Australia expects it.

It’s a stupid TV show. Its premise is bonkers. And it’s mainly some of the worst kinds of people who would want to be on it. They’ll never put an honest man on the show again – they’ve learned their lesson. From now on, every bloke playing the season’s bachelor will be an impossibly handsome gay guy with a wholly invented and scripted back story.

Living Green

8 September, 2018

A while ago, Make Hack Void was contacted by a certain Tracey B, who was interested in a collaborative art project thingy involving, hopefully, some LED lights evoking waves on a sea shore. I had been fooling about with Arduinos and NeoPixels, and I mostly approve of art in general, so I stuck my hand up. The result was – well, not as grand as I might have hoped. But it did work, at least.

Anyway. It seems someone in Norway wants to try something similar, on a slightly bigger scale, so I’ll write down here some stuff which hopefully might be useful to her. I belive she already works with breadboards and arduinos, so this will not be a detailed how-to.

Initial Impressions

Tracey had already sourced a couple of strings of individually addressable LEDs off the internets, which she brought to our initial meeting. But, had no idea how to go about making them light up, much less do some kind of seascape-y thing. With the aid of a magifying glass app, I examined them and determined that each led was powered by a WS2811 driver chip and that the string had three conductors: power, ground, and data. So, my initial reaction was “yep, I think I can make these things go”. I took one of the two strings home with me to see if I could make them at least light up.

This proved to be no problem.

There were other possibilities that might have made the project different. For instance, there are RGB strings of LEDS with four conductors: ground, R, G and B. Another possibility is common-anode. Instead of ground, there’s a +5v. This works well with a ULN2003A. You use a PWM arduino output to switch the ULN2003A, which supplies the power that the lights need. The disadvantage is that you only get one colour for the whole string, which limits what you can do. The advantage is that some of these strings are 12v and quite a bit brighter than what we had.

In any event, I found that the neopixel library that adafruit makes for the arduino worked perfectly fine, except that this particular string did not have the standard RGB arrangement. That also is perfectly fine: the library comes with 6 constants for the 6 possible permutations of RGB. It meant that mixing these particular lights with a string of the more standard variant would be awkward ÔÇŽ but we weren’t planning to do that.

Making them light up

So, with the aid of a loupe I worked out which end of the string was in and which was out. Or maybe I just tried it both ways and found which one worked ­čÖé . Ground and power off the arduino board itself into red and black and one of the standard adafruit demo sketches into pin 6 and from there into the white conductor on the string, and the thing lit up just fine. I discovered the RGB/BGR thing about this particular strand at that time, but this was no problem.

The code

The code – legal arrangement

When doing Arduino projects, I do so with the agreement that the code will be released into the public domain and published on github.

Even for paid projects, the code is released publicly. My position is that you are not paying for proprietary code, you are paying for getting me to write the code that you want. No contracts, no legal nonsense. When I accept jobs on the ‘gigs and collaborations’ board, I just ask that the fee be remitted to my PayPal if the code works for my client and they are happy with it. I think I have been done out of my fee only once.

For a weekend job, I typically charge $50 AUD. It’s my hobby, I do it for fun. I think I did Tracey’s job for free (pretty sure), and just asked her to cover the cost of parts. I do confidential, proprietary code at my real actual job. Any sort of confidentially requirements mean that you have to pay me real actual money, which for me is $100/hr.

Tracey was perfectly happy with the Living Green code being released to the public domain, which is what I expected, and we could proceed.

The code – design of the effect

The code is here.

The only design spec I was working with was “something something waves on the sea shore, it can be pretty abstract”. I didn’t know at the time how the strings were going to be mounted or arranged. The string itself was 50 lights long, and Tracey had two of them. I wrote a sketch that would power 100 lights.

Neopixels work by chaining the data. Data is sent to the lights in a stream, and each light removes the head of the stream and sends the rest of the data down to the next light in the chain. If the data falls off the end of the chain – this is no problem. That meant that I didn’t need to worry about the exact number of lights we would be working with.

So, I set up a system of sine waves that would be added together. It’s hideously over-engineered, of course, because that’s just how I roll.

The heart of the system is a C++ class named ‘Sine’. Sine has a wavelength and a frequency (fixed values), and a current theta position. Frequency is rotations per second, wavelenght is how many pixels long the wave is. When advanceMicros() is called, theta is updated. The v() method gets the current value of the sine wave at a given position along the strand, adjusted to a 0-1 range. v_for_mul gives a value of -1 to +1, which I decided to include when I wanted to multiply sine waves together.

There is additional code in there to permit the creation of a static ‘advanceAllMicros’ method. When the Sine objects are instantiated, they automagically add themselves to a static linked list of Sine objects. This means that if you simply create a Sine object and the code periodically invokes advanceAllMicros, that Sine object will move with correct timing without you having to do anything further.

With that class written, it was a matter of coming up with a pleasing effect.

I created a sine wave that moved in one direction (waveIn), and another that moved in the opposite direction (waveOut). I gave these a slightly different wavelenght and speed so that it wouldn’t look too regular. I decided that one of them would be green-ish, and the other kinda cyan. The rgb value of each pixel becomes an average of these two values.

I then decided that it would be nice if these two effects sort of took turns – the wave would move one way and then the other. So I created another sine wave, waveInOut, which moves a bit slower. Rather than taking an average of waveIn and waveOut, I took a weighted average, the weight being taken from waveInOut. This weight doesn’t vary by position along the string – I just use waveInOut.v(0). I could have used position along the string, but you want the effect to be ordered enough that a viewer can see a pattern, but no so ordered that it’s predictable. Another issue is that people aren’t going to be looking at it for very long, so it’s can’t be too visually chaotic.

Finally, I decided that it would be nice to have some sparkly highlights simulating ripples. So I made rippleBase and rippleMul. These run fairly quickly, and the effect is difficult to describe. But I was happy with it. I add this as a pure blue component to the final result, just to mix it up a little.

Finally, I felt that maybe a high-tide/low-tide component would be nice, so I added a tide Sine object with a frequency of ‘once every half hour’, which would adjust the overall effect of the main wave components.

The point of all this being: once you have defined an object that does something definite and simple, playing about with it is reasonably easy. With the Sine object defined, fooling about with different frequencies, different intensities, different combinations of waves is simpler.

What I would do differently

I’m pretty happy with the effect I eventually got. The main issue was the ‘tide’ thing. Having a half-hour cycle is kinda cool, but no-one is standing there looking at them for that long, so it was a bit pointless. The lights were not bright enough for the site, and having them slowly go dim over time didn’t work well on the day. My clients addressed this by rebooting the whole thing periodically, which isn’t ideal.

The code – overall control

Well, with all those parameters to play with, you’d like your client to be able to play with them. I built a mounting box for the project out of a small plastic food container and put a potentiometer and a couple of buttons in it. This allows you to get numbers into the sketch.

Ultimately, I rigged it up so that you could control the overall speed of the effect and the hightide/lowtide brighness. As I mentioned, the whole ‘tide’ thing was a bad idea, so ultimately this control box was pretty useless. I stuck the Arduino itself into it, but that’s all it turned out to be good for.

What I would do differently

These days, I’d provide a web interface. Several ways to do this.

Alternative ways to do sketch control

Don’t bother.

Why bother? It’s some background lights for the real show, which is the art stuff and art words. Relax! Just because I’m doing up some blinkenlights, doesn’t mean it’s the most important thing in the world. Get it right and put that in the code. Of course: this means that you really need to see the final result installed so that you can perform final code tweaks.

Arduino serves up web pages

This is not ideal if you want the general public to be able to fool with the settings, because network interaction will interfere with the display. It will freeze or get choppy while pages are being served.

There’s issues with storage, too. Web pages are bulky, especially if you want images. So we are talking SD cards. It’s really not worth it.

Arduino communicates via HTTP/JSON

Put the webpage elsewhere, have the arduino accept JSON packets from the HTML forms. I have had some sucess using AngularJS for this. The NodeMCU connects to the household web router, with the SSID and password being given to the arduino over a serial connection and stored in EEPROM. Most of the web page content is on wordpress or something.

For the ‘art installation in a shipping container’ situation ÔÇŽ perhaps you’d want to set up a router.

Arduino communicates via serial

That is, you have a computer hidden behind a cushion somewhere and an app running on it talking to the arduino over USB. I’m tending to think this would be the best option if you want to let the system communicate with the general public, simply because it allows you to use a domain name. Have the domain name point at a server on that computer, and have it connected via serial to the microprocessor. To keep people from fooling with the exhibit remotely, have a password that changes periodically.

In general

In general, it helps if you have a fairly definite idea of what you want a system to do. “It should have public interaction” needs to be narrowed down a bit.

Any system that involves serving up web pages is going to need something better than a microprocessor to actually serve them up. A microprocessor can do it, but not well.

I would not be inclined to attempt this remotely, communicating via skype or email.

Physical installation

So, the big day came. We were going to actually mount these lights into Tracey’s art thing.

The art thing turned out to be a fabric covering over some seashell forms made of bubble-wrap. (Which doesn’t do the art side of things justice, of course. Is the Mona Lisa just some pigment on a board? If your job is maintaining the Lourve climate control, then yes.)

The lights themselves – I should mention – were more like christmas lights than like the neopixel strips available from adafruit. Attached to the forms, they did not lay down in neat rows, which was good.

We decided to lay the lights on top of the forms under the fabric. We tried laying them under the bubblewrap, but the light was a bit too diffuse and dim. In the end, we simply stuck them down with tape.

Strings of neopixels can be cut. To operate, all that needs to happen is that they need to be supplied with 5v power, and a data line needs to come out of the out of the last pixel on one string and into the in side of the other. I was concerned that chopping the strings up into fragments might ruin the effect, but this turned out to be not the case. For the ‘nautilus’ shell, we attached a string in a spiral. For the scallop, we cut a length of string into five strips, lad them next to each other, and connected them. The ‘wave’ effect was still evident, or evident enough. Enough that you could say ‘yep, that is totally meant to be the sea’, which is all you need really.

Connections were done from the arduino to the pieces and between each piece using brown automotive wire and automotive ‘bullet’ connectors. This was very horrible, bulky, and ugly. I have since looked at using modular phone handset connectors for the same job, but these are also pretty horrible and the socket for a modular phone connector is bulky.

In retrospect, a three-conductor stereo audio cable with 1/4″ jacks might have been better. An issue might be capacitance, which would limit the length of the the data line.

You cannot power lights like this off the microprocessor pin. We used a separate power supply. The arduino and the neopixels both use 5v USB power, so any USB power-pack is suitable and proved adequate. Power was taken out of this for both the lights and the microprocessor.

Speaking of which, originally I had a UNO-3 set up for all this, but during installation I decided it was a little bulky. I had a leostick clone in my parts box, and soldered connections onto that. Much smaller.

So, how did it go?

It was just not bright enough. Originally, the exhibit was outdoors. Outdoors on an Australian spring day. It’s just not feasible to do any sort of light show it a situation like that. The Australian sun is not just bright, it’s full of UV which physiologically makes it even brighter. Fireworks are not bright enough on an Aussie spring day. 5v LEDs? They never had a chance.

The exhibitors managed to get it moved inside, which at least made the display visible. As I mentioned, the tidal stuff was a mistake, making the lights dim in comparison to ambient.

But, it was rather pretty overall. The power supply worked just fine, and the thing was nicely in context. A couple of people stopped and looked for a moment, which is mostly all you can hope for. Unappreciated art is almost proverbial.

I’m thinking of doing something like this, but in a shipping container. Do you have any advice?

Jeez. Shipping container eh?

I’m pretty sure a string of 50 5v LEDs isn’t going to cut it there, either. Not if it’s a major part of the installation. Here’s some 12v neopixels. Here’s some more. On the plus side, they’re 12v and bright as. On the minus side, it’s $50-$60 for 20 of them. You are not going to get away with much change from a thousand bucks if you want to have an effect encompassing the whole container, no matter how you do it.

You’ll want to inject power into them at intervals – don’t just try powering them from one end. The current will set fire to something. Power each strand of 12 individually. Data is less of a concern. You can daisy-chain everything you have, or run separate logical strands. Adafruit will happily sell you a power supply, but if you’re in Norway then you’d be better off just going to an electronics shop. The nice thing about 12v is: car battery.

I’d get online and search “outdoor stage lighting”, or “DMX”, or I’d head over to “Capital Sound and Lighting” (or the equivalent wherever you happen to be) and talk to those guys. Rent something. I’d explain the them that gosh darn it I’m an actual computer programmer and I can make DMX-driven lights dance, which means actually getting one or two from somewhere and playing with them beforehand so you can justifiably claim that.

An arduino will talk to DMX lights. But there might be better options. Although it’s unlikely that there’s be an off-the-shelf effect box that does what you want, quite possibly there’s a way to get some python code or something talking to an attached peripheral.

In summary

  • The best is the enemy of the good.
  • Some lighting won’t amaze anyone, not in this day and age. Remember that it’s only a support for the actual content of the exhibit.
  • What you are thinking of is probably do-able. It’s going to cost some money.
  • Someone like me needs something concrete to work with first. Art usually happens within the constraints of some medium. The key moment was when Tracey handed me the strand of lights that she wanted me to work with.
  • Nothing is ever quite as good or quite as bad as it first appears to be.


28 March, 2018

A Faustian bargain. A deal with the devil.

What is it, precisely, that makes a Fasustian bargain Fasustian? Any bargain involves exchanging X for Y, and that’s cool. But a Faustian bargain is more than simply an unfair or ill-considered bargain. It’s more than simply that X was actually worth more than Y. It’s that X was the thing that gave Y its value. That when you surrender X to get Y, Y becomes worthless because you have surrendered┬áX.

Wilde (I think) does this in “The Gift”. The classic is the exchange of your soul for power. Once you have surrendered your soul, any power you might wield you only wield at the command of the one to whom your soul now belongs.

Awesome as Y might be, if you surrender the entire point of having Y in order to get it, then you have made a bad bargain.

More Minecraft

4 March, 2018

This is going to get very freaky, very weird. Oh, maybe not for people who already have a particular kink – people with lisps, Haskel programmers. But everyone not already an FP weenie will need to take a shot of espresso and/or whiskey.


So, the build described on my previous post does indeed process astral sorcery crystals. But building all of those chains of NBT variables is a pain. Wouldn’t it be nice if there was a function that would simply get the size and cut of a crystal, nested all the way down in those NBT tags? Well, you can build them.

There are a couple of things to know. First, all of the built-in operators can be turned into an operator variable using (obviously) the operator entry in the portable programmer. You can then use the apply operator to give that variable some parameters, and it works the same as if you had simply used the operator.

Second, FP languages have an interesting method of handling functions with multiple arguments. In principle, a function only ever has one argument – it maps one thing onto another thing. So how is something like, say, addition done? Say we want to add 100 and 200. Well, the + function takes one integer argument. So when you give it a parameter of 100, it returns a function that adds 100 to things. This function is then applied to the second argument, giving you the 300 you wanted.

In Integrated Dynamics you can totally do that, you can take a function taking three arguments and bind something to the first argument, giving you a function (an operator) that takes two arguments. And so on.

I suppose the third thing to note is that all the things I am interested in start at a tag named ‘tag’. That is, various things give you back various types of lists and NBTs. The outer layers are specific to entities, inventories, blocks. For instance, an entity reader gives you an NBT that talks about the location and orientation of the entity, the inventory reader gives you an NBT talking about which slot in the inventory an item is in, the grindstone just starts at the top level, because it’s not done as a cointainer. The stuff common to these containing the astral sorcery gear starts at ‘tag’, wherever that might be. From that point you walk down: tag, astralsorcery, crystalProperties, size/collectiveCapability/purity.

So what I really want is a pair of functions which, given an NBT with a ‘tag’ entry, walk down to the integer ‘size’ entry and return it.

Let’s see how deep the rabbit-hole goes.

Walking down the tree

First, create an NBT object with that ‘tag’ at the top level. For instance, if your entity reader has a crystal floating in front of it, grab the entity aspect of the reader, then use entityNBT and then NBT.tagNBT to grab the ‘Item’ entry out of that tag. You could build a function to do this, but for now I am not going to bother.

I label the item variable ‘crystal’, its NBT ‘crystalNBT’, and the Item entry of that ‘crystalItem’.

This gives us an NBT with the structure that we will be working with – it’s our test data.

Next I use the portable programmer ‘Operator’ function to get two operators: ‘reduce’ and ‘NBT NBT Value NBT’. Yeah, I know. I label these ╬╗reduce and ╬╗getNBT.

I then create a list of string containing the values ‘tag’,’astralsorcery’, ‘crystalProperties’. Case-sensitive, remember. I label this tag2cp.

Now at this stage, we can get the inner NBT fairly easily without needing ╬╗reduce. Create a variable ‘reduce(╬╗getNBT, tag2cp, crystalItem)’. This will use the reduce function to walk down the list of strings applying the ╬╗getNBT operator each time, starting with crystalItem. At the end of the list, it will have reached the target NBT.

You can achieve the same effect using the apply3 operator: ‘╬╗reduce apply3(╬╗getNBT, tag2cp, crystalItem)’.

But I don’t want a variable that extracts the stuff out of crystalItem, I want a function that will extract out of any suitable NBT.

Making the treewalk into an operator

To do this, I ‘╬╗reduce apply(╬╗getNBT)’ to get another variable which I name ‘╬╗nbtWalk’. This is a function that takes two arguments – a list of string and an NBT. It will walk down any NBT with a list, returning the final NBT.

I then create a variable ‘╬╗nbtWalk apply( tag2cp)’, which I label ‘╬╗nbt2cp’. This is a function which, given an NBT, will return that final NBT containing the crystla properties (if it’s there, obviously).

When you create a variable ‘╬╗nbt2cp apply(crystalItem)’ and drop it into a display, well blow me down if it don’t display that inner NBT containing the stuff we want. Let’s label this variable ‘cprops’ for now. We wont need it in the final result, but it’s handy for testing the next bit.

Getting the integer

Now, we can’t use the ╬╗nbtWalk function to get all the way down to size and cut, because these are integers and the ╬╗getNBT function will return nothing.

So, we create an operator variable out of ‘NBT NBT Value Integer’ and label it ‘╬╗getInt’. You can apply an NBT and a string to this to get an integer from inside an NBT. The problem is that the arguments are around the wrong way. If you used ‘apply’, you’d have to bind the NBT first and you’d get a function that would get any integer out of some particular NBT if you give it a tag name. NOt what we want.

So, create a new operator using ‘flip(╬╗getInt)’. This gives us the get Int function with its arguments swapped. I name this ‘╬╗getInt_YX’, with the idea that function arguments are usually named ‘X’, ‘Y’ in that order.

Create a string ‘size’ labelled – ohh, I dunno – ‘size’. At this stage, you can create a variable ‘apply2(╬╗getInt_YX, size, cprops)’ and drop it in a display, and you should see an integer. And not just any integer, but the size of the crystal sitting in front of the entity reader.

Now create a new variable ‘╬╗getInt_YX apply(size)’. This gives you a function returning the size value from any NBT that it is passed. I label this ‘╬╗getSize’.

Putting it together

And to put it together, we use the pipe operator: ‘.’. Simply ╬╗nbt2cp . ╬╗getSize’ to give a new operator, which I label ‘╬╗nbtSize’. And this is the magic, this is the thing we are trying to build. It’s a function taking one argument – the top level astral sorcery NBT containing a ‘tag’ entry, and returning the size of the crystalProperties NBT buried three levels deep. Try it! Create a variable ‘╬╗nbtSize apply crystalItem’. Drop it in a display, and you should see the size. Naturally, you will want to make ones for purity and collectiveCapability (which I always label ‘cut’) as well.

From here

From here, you can, well, do stuff. Turn ‘entity->get(‘Item’)->╬╗nbtSize’ into a single operator, if you want.

But the real thing is building predicates. With these functions, you can now build a predicate ‘crystal has size 900’ by applying 900 to the == operator and piping ╬╗nbtSize into it. This can be used in item exporters and importers as a filter. Stuff like that. I’m not going to rebuild my setup using this method, but the business of sorting out completed crystals from ones still needing to be cut could have been done that way. If it were the case that you could put multiple crystals in the one growth pool, and you want to pull out the ones that are ready, then that’s the way to go.

Anyway, that’s what it took to make this happen in my creative test world. Haven’t used it in survival, yet, but that’s how it’s done.

Some minecraft

2 March, 2018

This post attempts to describe my setup for growing maxed-out astral sorcery crystals using integrated dynamics and integrated tunnels.

To make all this work, we use several ID channels. Each channel has a single chest or fluid tank. The components work by exporting and importing crystals between appropriate channels.


Run fluid pipe from beneath your lightwell into a tank. If you are using Thermal Dynamics for this, it must be at least hardened or it will break (starlight is extremely cold). You will need a servo, because lightwells don’t auto-extract. Attach a fluid interface to the tank – my setup uses channel 0.


Add three chests to the network each with its own item interface on channels 1,2, and 3. Channel 1 will be for crystals needing to be grown, channel 2 for crystals needing to be cut, and channel 3 for completed maxed-out crystals.

Budding off new crystals

Create a spot for the budding pool. This spot will have an entity reader, a world fluid exporter, and an world item importer. Easiest way to to this is to place a solid block, put the components on that, then break the solid block.

Create a fluid variable from a bucket of starlight. This goes in the place fluids aspect of the fluid placer, using channel 0. This will keep the block full of starlight as it is used up.

Throw a celestial crystal with a purity of 100% in the pool. To get one of these, go through the process manually: you only need to do it once.

Make a variable from the ‘entities’ aspect of the reader. This will be of list type. Make an integer variable from the len of that list. Make a constant variable 1. Make a boolean variable that is true when the list length > 1. Put this in the ‘pick up all item entities’ aspect of the world item importer. Set the channel to 1 (or 2, that also works). The rate does not matter, because each crystal counts as its own stack.

The result is that the crystal in the pool will be grown, the starlight replenished as needed, and when a new crystal buds off, it will be put in the chest on channel 1.

Growing the crystals

Make another pool. This one needs a world fluid exporter exporting starlight from channel 0, an entity reader, a world item exporter, and two world item importers.

As with the budding-off pool, read the entities aspect to get a list variable and use the ‘is empty’ list operator to get a boolean variable (easier than len()==0). This goes in the ‘place all item entities’ aspect of the world item exporter, which must get its items from channel 1. The effect of this is that it will pick out a crystal from the ‘needing to be grown’ chest whenever there is no crystal in the pool.

And now it gets a little complicated. ­čÖé

Take a variable off the ‘entity’ aspect of the reader. Make a variable entity off this using ‘entity NBT’, to give an NBT object. Create String variables with the strings ‘Item’, ‘tag’, ‘astralsorcery’, ‘crystalProperties’, ‘size’, and ‘collectiveCapability’. These are probably case sensitive.

Using the NBT.tag function, create a series of variables drilling down the NBT tree -> ‘Item’, ‘tag’, astralsorcery’, ‘crystalProperties’, then on that final NBT create two integer variables using NBT.integer for ‘size’ and ‘collectiveCapability’. Use your labeller. In my setup, I label the ‘collectiveCapability’ variable to ‘cut’, and the ‘size’ variable to ‘size’, duh.

Create two integer variables with values 100 and 900 (assuming we are making celestial crystals. You’d use 400 for regular crystals ÔÇŽ but why would you bother? You only need one to start with). Create three boolean variables size==900 (which I label ‘big’), cut==100 (which I label ‘perfect’), and cut!=100 (which I label ‘imperfect’).

Create a variable using the && operator for ‘big && perfect’. This goes in the ‘pick up all item entities’ aspect of one of the world item importers and should import into to channel 3. This is the output.

Create a variable for ‘big && imperfect’. This goes in the ‘pick up all entities’ aspect of the other world item importer and should import into channel 2.

The end result is that whenever this pool is empty, a crystal will be placed into it from chest #1 and the crystal will be grown until it has size 900. At that point it will get put either in chest 2 or 3 depending on whether is it properly cut or not.

Cutting the crystals

Place a grindstone. This will need 3 player simulators and a block reader. Place another chest with an inventory reader, an item interface, and an item exporter. The item interface will be on channel 4.

Take a variable from the ‘inventory empty’ aspect of the inventory reader. I label this ‘4empty’.

Take a variable from the Tile Entity NBT aspect of the block reader on the grindstone. Make string variables ‘tag’, ‘astralsorcery’, ‘crystalProperties’, ‘size’ and ‘collectiveCapability’ (or re-use the ones you already have).

As with the growth pool, descend the NBT tree on the grindstone to get integer variables for tag.astralsorcery.crystalProperties.size and .collectiveCapability . I prefix all the label names with gs_ for ‘grindstone’, and so these two variables get labelled ‘gs_size’ and ‘gs_cut’.

Use the NBT.hasKey(‘tag’) operator on the root tile entity NBT variable to make a variable named ‘gs_hasCrystal’, and the negation of this to make a variable named ‘gs_noCrystal’. This works because the ‘tag’ NBT entry on the grindstone is only present if there is a crystal in it (there is no ‘Item’ entry because the grindstone is not an inventory – that’s why you need player simulators).

Make a boolean variable labelled ‘gs_perfect’, from ‘gs_cut == 100’. Make another boolean variable ‘gs_imperfect’ from ‘gs_cut != 100’.

Make a boolean variable ‘gs_toosmall’ from ‘gs_size < 100'. I use 100 because a single pass of the grindstone takes less than 100 size off a crystal, and I already have an integer variable with a value 100, so I can re-use it ­čÖé .

Make a boolean variable 'gs_done' from 'gs_toosmall || gs_perfect', and a variable 'gs_notdone' from '!gs_done'.

On the item exporter on the chest, have it 'export all items items' from channel 2 with the condition 'gs_noCrystal && 4empty'. This will place one and only one crystal needing to be cut in the chest when the grindstone is empty. Because the item interface on the chest is channel 4, and the item exporter takes from channel 2, the crystal is moved from channel 2 to channel 4 of the system.

Make an 'item' variable from any celestial crystal.

On the first player simulator on the grindstone, have it 'click item' using the celestial crystal item variable. Channel 4, right-click true, and check NBT to false (otherwise the crystal needing to be cut must be identical to the crystal you made the item variable with). This will place a single crystal from chest 4 into the grindstone, but because the chest only has a crystal when the grindstone is empty, it will not continuously right-click the grindstone and potentially ruin the crystal in there. In effect, what is happening is that chest #4 is used to overcome the problem that we cannot put a boolean on the 'right click with item' aspect of the player simulator. The presence or not of a crystal in chest 4 serves as that boolean.

I have just been watching direwolf’s youtube video. I was not aware of the tiny chest. Using one of these would make this system simpler, as you do not need the 4empty mechanism to ensure that the chest only has one item in it. You still need the gs_noCrystal so that it only gets filled when the grindstone has no crystal in it.

On the second player simulator, make a variable ‘gs_hasCrystal && gs_notDone’ and put it in the ‘click empty’ aspect. This will turn the grindstone until the crystal is too small to grind anymore or the cut is 100.

On the third player simulator, make a variable ‘gs_hasCrystal && gs_done’ and place it in the ‘click empty’ aspect with sneak == true. Set the channel to 1. This will pick up completed (or incomplete cut too-small) crystals and put them back in the bin of crystals to be grown.

The end result is that whenever the grindstone is empty, a crystal is right-clicked into it from chest 2 via chest 4. This gets empty clicked until the crystal is either perfect or too small to grind anymore. At that point, the third player simulator sneak-clicks it back into inventory #1, from where it will make its way to the growing pool.

General notes

And that’s it.

I use a separate variable store for each of the three processes – they’re not that expensive, and it helps with keeping things organised. Labelling the variables is also indispensably helpful when using the programmer. Label the strings, too.

Playing New Horizons III, I find that ID occasionally jams. This tends to happen when I reload the game, and when I AFK away at my main base which is several chunks away (my AS setup is in a sparkly star zone). To fix this, my variable stores are connected to the rest of the system via an extra length of logic cable. I find that breaking and then replacing it un-jams things.

I sometimes find that the World Item Importer on the budding pool fails to import an item. This leaves two crystals in the pool, and astral sorcery will only grow crystals if there’s just one. To fix, pick up both crystals and chuck one back in the pool. The other you can just put in chest 1.

Growing crystals is slow, but this system means you can AFK and come back to three or four maxed-out celestial crystals. One of these will make 11 lenses. The waste crystals that you got from creating your initial 100% pure crystal can be thrown into the lightwells.

And you can add more pools for growing and budding by making a new variable store, making a new set of variables for the pool, and importing/exporting to the existing chests. I’m finding that neither growing nor budding is obviously the critical path – budding is slow because once the crystal gets to 900, it will still be a couple more cycles before the crystal buds; but growing is also slow because after the crystal is grown to 900, cutting will reduce its size and it will need more growing anyway.

In the image at the top of this post, crystals are moving from right to left through the system, because that’s how I wound up building it.

The display reading 799 shows the size of the crystal in the budding pool. It will hit 900 and stay there until the crystal splits, at which point one of the smaller crystals will be removed and one will remain to be grown up to 900 again.

The displays reading 824 and 100 are the size and cut of the crystal in the growing pool.

Not visible in the image are two more displays showing the size and cut of the crystal in the grindstone. These are normally 0, because grinding happens very fast and the grindstone is usually empty.

The displays are not necessary, but they are kinda cool. They are also helpful in that you can see if the system is jammed or not – when it becomes jammed, the displays read 0 even though there should always be a crystal in the budding pool.

If you don’t want to build the full system, I’d suggest that the grindstone displays make it so much easier to grind crystals. Player simulators are expensive, so doing the grinding manually is an option – just come back to the setup every now and again and grind the crystals in chest 2. The displays make it a simple matter to just right-click that grindstone until the numbers are improved, and then it’s back into chest 1.

Personally, I think AS should be nerfed with the rule that if you bud off a new crystal from a celestial crystal, the new, purer crystal should just be a regular one. But as it is, with this system you can have a chestful of maxed crystals AFK.