Cross-device Compatibility for SparkSL and Shader Patches

The effects you create are likely to be used on devices with a range of different specifications. In this article, we’ll look at methods for ensuring the shaders in your effects work well across as many different devices as possible.

Shader precision

The underlying graphics library that runs your desktop computer (OpenGL) is different from those running many mobile devices (OpenGL ES / Metal). While they are similar in most respects, one important difference is the level of precision available in numerical values associated with shaders.

OpenGL ES exposes a precision modifier that allows control of the desired precision for different data types: highp, mediump and lowp.

When used on floats, the mapping is typically as follows:

  • highp - generally 32-bits (on some devices 24-bits in fragment shader)
  • mediump - generally 16-bits
  • lowp - generally 11-bits

The limits vary with the width of the floating point type:

  • 32-bit float - exponent range [-126, +126], relative precision 2^-23
  • 24-bit float - exponent range [-62, +62], relative precision 2^-16
  • 16-bit float - exponent range [-14, +14], relative precision 2^-10
  • 11-bit fixed float - range (–2, +2), absolute precision 2^-8

The precision modifiers are ignored on desktop computers, but will have an impact on mobile device rendering.

Vertex shaders use highp by default in order to guarantee proper vertex transformations. However, on fragment shaders this can sometimes cause problems. Currently not all devices support highp in the fragment stage, and when they do performance may still be worse than using mediump.

If your effect does not require highp, it is advisable to use mediump instead.

Avoiding shader precision issues

As mentioned above, using mediump in the fragment shader will limit the range of your values: :

  • 16-bit float - exponent range [-14, +14], relative precision 2^-10
  • Smallest representable number - 2^-14 = 0.000061
  • Biggest representable number - 2^+14 = 16384

With this limitation, it can be challenging to keep shader values inside of the acceptable range. Functions like x^2, pow, length and normalize will need to be used with care.

For example, you may encounter problems with:

  • length(xyz) = sqrt(x^2+y^2+z^2)if any component is larger than 128.
  • Big constant values, sometimes used for random number generation

However, there are some things you can do to mitigate these issues.

Before normalizing a vector, divide it by its L1 (aka Manhattan/taxicab) norm:

  • normalize(v / max(abs(v.x) + abs(v.y) + abs(v.z), 0.0001)) will bring the exponent close to 0 which is the best one can hope for when it comes to floating point precision.
  • A similar thing can be done for length. In SparkSL this functionality is already exposed with safeNormalize and safeLength.

When multiplying several values, reorder them so that the intermediate results are within the range:

  • For example it is better to do big1 * (big2 * small) than just big1 * big2 * small. In this way we may avoid an overflow on big1 * big2 intermediate result.

Use noise textures instead of generating noise procedurally:

  • It is very challenging to generate good looking noise with only 16-bits of precision. Textures will also be more performant

OpenGL undefined behaviour

When using visual shaders, we are abstracting the use of the underlying graphics libraries. In this case, OpenGL.

While there is no need to learn OpenGL, it is important to know that some operations are not well defined for all inputs. This means that the shader will continue to work, but the output of that specific operation can be unpredictable.

Here there is a list of operations that have undefined behaviour. This is true both for patches or functions on SparkSL.

  • pow(base,exponent) - undefined if base < 0, or base = 0 and exponent <= 0
  • log(x) / log2(x) - undefined if x <= 0
  • sqrt(x) - undefined if x < 0
  • inversesqrt(x) - undefined if x <= 0
  • asin(x) - undefined if abs(x) > 1
  • acos(x) - undefined if abs(x) < 1
  • atan(y, x) - undefined if x = 0 and y = 0
  • clamp(x, minValue, maxValue) - undefined if minValue > maxValue
  • smoothstep(edge1, edge2, x) - undefined if edge1 > edge2

In most cases, the problem is solved by ensuring that the input values are within the valid range. In cases where that cannot be guaranteed, it is recommended that you enforce it using operations like clamp, min or max.

Conditional invoking of texture sampling and derivatives

Shader Derivative functions may exhibit undefined behaviour if they are not applied to every pixel on the fragment shader (ie. if your function depends on a per-pixel condition).

This also affects the sampling of textures with implicit level of detail because implicit level of detail is internally computed by using shader derivatives. This means that if we sample a texture or compute a derivative value then we will need to do so for all relevant pixels, even if they are not used.

This code would show that undefined behaviour, since it is sampling a texture only for certain pixels:

if (color.a < 0.5)
  out_color = sample(uv);
else
  out_color = vec4(1.0);

To avoid issues, we could rewrite it in a way that the sampling happens on every pixel evaluation. The produced color should be the same as the previous code snippet:

float mix_factor = step(0.5, color.a);
out_color = mix(sample(uv), vec4(1.0), mix_factor);

Derivatives are exposed both in SparkSL and when using patches. It is important to follow these guideline in both scenarios.

Alpha output

Alpha isn't only relevant in the context of texture assets and color values - the image generated when rendering the effect has an alpha channel as well.

Most effects have a resulting alpha channel of 1.0 (totally opaque), but when they don’t it is left to the platform where the effect is being used to decide how to blend the image. Ensuring a correct alpha output value will result in future-proof effects as more platforms become available for Spark AR effects.

If your effect is already computing the final composited color, you can enforce alpha to be 1.0. This is particularly important in a custom render pipeline with visual shaders. You can visualize the resulting alpha with a single swizzle patch (aaa1), and then reuse it to enforce alpha to 1.0 (xyz1).

Delay Frame patches and undefined colors

Under some circumstances, the internal buffers of the Shader Render Pass patch can be reset. This can result in single frames which contain an undefined color. Undefined colors may happen when underlying OpenGL context changes or when the texture is resized, but these cases are outside of user control.

Although this by itself is not an issue, using computed textures in an undefined state can cause visual artifacts.

One case where this can become a problem is while using a reentrant image on a shader render pass where the output of the shader (delayed frame) is used as an input (receiver):

The underlying problem lies on the reentrant image being used during the first frame computation, which is the one that may be affected after the recreation of the internal buffers. Getting rid of the dependency on the first frame should be enough to avoid this problem.

Where more complex patch graphs are involved, this problem might not be so easily diagnosed. To quickly and easily recreate all internal shader buffers and check your effect for this issue, simply switch between Landscape and Portrait mode on the Simulator. This will force Spark AR to recreate all elements within the scene.

Was this article helpful?