> ... when you want to work with SIMD or GPU operations, one can (and we will) use a representation of arrays of Nullable objects that places the values and the Boolean flags into two separate arrays.
"When" is pretty much "always" with arrays. Trusting a sufficiently smart compiler and runtime to get rid of the tag checks makes me nervous, though. This optimization needs to be user-predictable and/or -controllable to be truly useful. If only compiler gurus can understand when slow, tag-checking, scalar loops will be replaced with fast SIMD code, something has gone wrong.
We will probably be pushing people very heavily towards always using specialized arrays for working with nullable objects.
I'm a little surprised by your comment about sufficiently smart compilers, since Julia seems to have taken the exact opposite approach to compiler design. The language is systematically designed to require little intelligence from the compiler. Being able to safely drop tag checks inside of a function's body seems like a fairly simple part of the language's design, since the type of all inputs to a function and all function-local variables are, in general, easily knowable as invariants before a function call even begins evaluating.
> ... when you want to work with SIMD or GPU operations, one can (and we will) use a representation of arrays of Nullable objects that places the values and the Boolean flags into two separate arrays.
"When" is pretty much "always" with arrays. Trusting a sufficiently smart compiler and runtime to get rid of the tag checks makes me nervous, though. This optimization needs to be user-predictable and/or -controllable to be truly useful. If only compiler gurus can understand when slow, tag-checking, scalar loops will be replaced with fast SIMD code, something has gone wrong.