Particles wander at random until they bump into something, then stick there permanently and become part of what the next particle can bump into. From a single seed, coral grows.
Why it branches instead of filling in solidly is the interesting part. A wandering particle is far more likely to run into a tip sticking out into open space than to find its way down into a crevice between two existing branches, because the branches get in the way first. So every protrusion grows a little faster than every hollow, and that small advantage compounds relentlessly. Nothing tells it to make branches. Branches are what shadowing does to random walkers.
This process is called diffusion-limited aggregation, and it shows up in electrodeposits, mineral dendrites, lightning and frost on a window, for exactly the same reason.