Loris Cro

Zig has just gained new for loop syntax that allows you to iterate on multiple slices / arrays at the same time. In this blog post I’m going to explain in detail the rationale behind this choice, while also introducing you to a couple useful patterns that the syntax is meant to encourage.

If you want to try it out, you will need an unstable build of Zig, which you can get from the downloads page.

Basic syntax

The most basic for loop syntax in Zig is still the same as before.

const elems = [4]usize{ 10, 20, 30, 40 };

for (elems) |x| {
   std.debug.print("{} ", .{x});
}

This prints:

10 20 30 40 

If you’re new to Zig, you might be surprised by the |x| syntax. That’s called a capture in Zig and in the case of for loops is how you can, well, capture the iteration value and give it a name.

Ranges

The new syntax also supports ranges, which are a new construct in Zig.

for (0..4) |n| {
   std.debug.print("{} ", .{n});
}

This prints:

0 1 2 3 

Ranges can also start from something other than zero.

for (1..5) |n| {
   std.debug.print("{} ", .{n});
}

This prints:

1 2 3 4 

Ranges can only exist as an argument to a for loop. This means that you can’t store them in variables, but you can use variables to specify their bounds.

var a: usize = 10;
var b: usize = 15;

for (a..b) |n| {
   std.debug.print("{} ", .{n});
}

This prints:

10 11 12 13 14

Multi-sequence syntax

The new multi-sequence syntax allows you to loop over two or more arrays or slices at the same time:

var elems = [_][]const u8 { "water", "earth", "fire", "air" };
var nats = [_][]const u8 { "tribes", "kingdom", "nation", "nomads" };

for (elems, nats) |e, n| {
   std.debug.print("{s} {s}\n", .{e, n});
}

This prints:

water tribes
earth kingdom
fire nation
air nomads

There’s only one simple rule when it comes to the length of the sequences: all lengths must match. Passing arrays of different length is safety-checked UB (i.e. you will get a panic in safe release modes).

Ranges as indexes

To iterate over a sequence and also keep track of the element’s index, you can add a range to the list of sequences you want to iterate. Since all sequences must have the same length, you can omit the upper end of the range and let Zig automatically infer it from the other sequences.

var elems = [_][]const u8 { "water", "earth", "fire", "air" };
var nats = [_][]const u8 { "tribes", "kingdom", "nation", "nomads" };

for (elems, nats, 0..) |e, n, idx| {
   std.debug.print("{} - {s} {s}\n", .{idx, e, n});
}

This prints:

0 - water tribes
1 - earth kingdom
2 - fire nation
3 - air nomads

Other properties of for loops

Up until now we saw the new changes to for loops, but if you’re new to Zig you might not know all the other things they support, so I’ll quickly recap them in this section.

Pointer to the element

Value captures in Zig should always be understood as immutable copies. To ask for a pointer you can add a * before the capture name.

var good_digits: [3]usize = .{4, 2, 0};

for (&good_digits) |*d| {
   d.* = 6;
}

// for (good_digits) |d| {
//    d = 6;
// }
//
// error: cannot assign to constant
//     	d = 6;
//     	^

Labels, break and continue

You can give labels to loops, which helps breaking and continuing iteration at the right level.

const vowels = "aeiou";
const text = "lorem ipsum";
var missing = false;

outer: for (vowels) |v| {
   for (text) |x| {
      if (x == v) continue :outer;
   }
   missing = true;
   break :outer;
}

else for for loops

In Zig you can give an else branch to a for loop. The else branch triggers when the loop ends naturally, as opposed to breaking from it.

This models beautifully searching for an element in a sequence: if the element is found, you will break from the loop, while if it’s not found then the loop will end naturally, at which point the else branch will allow you to implement the “not found” case.

for loops can also be used as expressions, which works particularly well in this case.

const text = "abcdef";
const needle = 'e';

const match: ?usize = for (text, 0..) |x, idx| {
   if (x == needle) break idx;
} else null;

Inlined for loops

It’s possible to operate on heterogeneous sequences of values with for loops when doing comptime metaprogramming. You can learn more in this old blog post of mine.

Multi-sequence for loops and data oriented design (DOD)

Say that you have a game where each monster has an element type, a counter for hit points, and a unique “dna” string used to procedurally generate stats for each monster’s offspring (and to give an early taste of how it feels to play with slot machines to young kids).

const Monster = struct {
   elem_type: enum{ fire, water, wind, earth },
   hp: usize,
   dna: [33]u8, // gambling department demands 
                // we use exactly 33 bytes
};

First of all, you would probably want the in-memory representation of this struct to place hp at the top of the struct in order to avoid the need for padding inside the struct to maintain its natural alignment (because its type is usize, which has 8 byte alignment on common 64bit machines).

Luckily, this is done automatically by Zig (you can use a extern struct if you want field ordering to work like in C), but even then, the struct has alignment 8 and size 42, which means that it needs 6 bytes of padding at the end to keep the alignment consistent in an array (ie @sizeOf([2]Monster) == 96), so in the end some padding is inevitable given the size of our fields. 6 bytes might not seem much, but it does mean that each monster instance wastes roughly 15% of its size just for padding.

One technique that lets us prevent that waste is to avoid representing our monsters as an array of structs (AoS), and instead “deconstruct” them into multiple arrays, one per field (also known as struct-of-arrays, SoA):

monster_hps: []usize,
monster_dnas: [][33]u8,
monster_elem_types: []enum{ fire, water, air, earth },

This memory layout wastes no bytes and also lets us operate more efficiently on our data.

Let’s say that fire monsters gain one hit point every tick of our game. This means that every tick we want to look at each monster’s elem_type and based on that we increment its hp by one.

If we were to do this with the original array-of-structs layout, for each monster we would have to load from memory 39 bytes of data that we don’t care about (33 from the dna field, plus 6 of padding) over the 9 bytes that we do need. That’s a waste rate of more than 400%!

With the struct-of-arrays layout we only load from memory data that we do care about, which can have a tremendous effect on performance.

for (monster_elem_types, monster_hps) |et, *hp| {
   if (et == .fire) hp.* +|= 1; // saturating addition
} 

The Zig standard library has MultiArrayList, a data structure that helps make DOD style programming even more ergonomic. Here you can read more about it.

Hoisting safety checks

In low-level programming languages, accessing an array item corresponds to adding an offset to a pointer value and then dereferencing it. This operation is very fast but, if the logic is wrong, one could end up reading past the end of an array and the program wouldn’t even notice.

In Zig out-of-bounds array accesses are safety-checked in safe build modes which means that the compiler adds a hidden assertion whenever an array access is about to happen.

var idx: usize = 5;
assert(idx < my_slice.len); // secretly added by the compiler
_ = my_slice[idx];

If we were to implement the previous game feature (fire monsters getting 1 hp every tick) without multi-sequence for loops, we would have to do something like this:

var idx: usize = 0;

while (idx < monster_count) : (idx += 1) {
   const et = monster_elem_types[idx]; // potential oob
   const hp = &monster_hps[idx]; // potential oob

   if (et == .fire) hp.* +|= 1;
} 

Unfortunately, the Zig compiler would have to insert two hidden assertions with this version of the code: one before the assignment to et, and one before the assignment to hp.

In the multi-sequence for loop version it’s only necessary to test once at the beginning of the loop that the two arrays have equal size, instead of having 2 assertions run every loop iteration. The multi-sequence for loop syntax helps convey intention more clearly to the compiler, which in turn lets it generate more efficient code.

Of course, with sophisticated-enough static analysis the compiler could prove that monster_count is always equivalent to monster_elem_types.len and monster_hps.len, and thus it could elide the assertions, but static analysis slows compilation times and tends to be a fragile thing, like this amazing blog post about loop optimizations in C# shows.

Multi-sequence for loop syntax doesn’t slow down compilation times and guarantees that you get good performance also in debug builds, where advanced optimizations are disabled and compilation times matter the most.

Conclusion

Zig is already a compelling programming language and toolchain, but there’s more design space to explore ahead of us before we can tag v1.0.

We recently self-hosted the compiler and optimized our development process in order to make it as smooth as possible to explore new design ideas like multi-sequence for loops.

Zig describes itself as a programming language for maintaining robust, optimal and reusable code, and multi-sequence for loops are a shining example of how the language tries to strike a compelling balance between clarity, performance and safety.

Comptime metaprogramming allows us to have std.MultiArrayList, a userland implementation of AoS/SoA transformation, which makes it easier to do DOD-style programming, while multi-sequence for loops ensure that we get all the safety of out-of-bounds checks without impacting runtime performance nor compromising on compilation times.

If you like where we’re going, please consider sponsoring the Zig Software Foundation.