Rust GLSL crate

It’s been almost two months that I’ve been working on my glsl crate. This crate exposes a GLSL450 compiler that enables you to parse GLSL-formatted sources into memory in the form of an AST. Currently, that AST is everything you get from the parsing process – you’re free to do whatever you want with it. In the next days, I’ll write a GLSL writer (so that I can check that I can parse GLSL to GLSL…). I’d love to see contributions from Vulkan people to write a SPIR-V backend, though!

Just for the record, the initial goal I had in mind was to parse a subset of GLSL for my spectra demoscene framework. I’ve been planning to write my own GLSL-based shading language (with modules, composability, etc. etc.) and hence I need to be able to parse GLSL sources. Because I wanted to share my effort, I decided to create a dedicated project and here we are with a GLSL crate.

Currently, you can successfully parse a GLSL450-formatted source (or part of it, I expose all the intermediary parsers as well as it’s required by my needs to create another shading language over GLSL). See for instance this shading snippet parsed to an AST.

However, because the project is still very young, there a lot of features that are missing:

• I followed the official GLSL450 specifications, which is great, but the types are not very intuitive – some refactoring must be done here;
• I use nom as a parser library. It’s not perfect but it does its job very well. However, error reporting is pretty absent right now (if you have an error in your source, you’re basically left with Error(SomeVariant) flag, which is completely unusable;
• I wrote about 110 unit tests to ensure the parsing process is correct. However, there’s for now zero semantic check. Those are lacking.
• About the semantic checks, the crate is also missing a semantic analysis. I don’t really know what to do here, because it’s a lot of work (like, ensure that the assigned value to a float value is a real float and not a boolean or ensure that a function that must returns a vec3 returns a vec3 and not something else, etc.). This is not a trivial task and because this is already done by the OpenGL drivers, I won’t provide this feature yet. However, to me, it’d be a great value.

If you’re curious, you can start using the crate with glsl = "0.2.1" in your Cargo.toml. You probably want to use the translation_unit parser, which is the most external parser (it parses an actual shader). If you’re looking for something similar to my need and want to parse subparts of GLSL, feel free to dig in the documentation, and especially the parsers module, that exports all the parsers available to parse GLSL’s parts.

Either way, you have to pass a bytes slice. If your source’s type is String or &str, you can use the str::as_bytes() function to feed the input of the parsers.

Not all parsers are exported, only the most important ones. You will not find an octal parser, for instance, while it’s defined and used in the crate internally.

Call to contribution

If you think it’s worth it, I’m looking for people who would like to:

• write a GLSL to SPIR-V writer: it’s an easy task, because you just have to write a function of the form AST -> Result<SPIRV, _>, for being pragmatic;
• test the crate and provide feedback about any error you’d find! Please do not open an issue if you have an error in your source and find “the error from the parsers is not useful”, because it’s already well established this is a problem and I’m doing my best to solve it;
• any kind of contributions you think could be interesting for the crate.

Happy coding, and keep the vibe!

luminance designs

luminance-0.7.0 was released a few days ago and I decided it was time to explain exactly what luminance is and what were the design choices I made. After a very interesting talk with nical about other rust graphics frameworks (e.g. gfx, glium, vulkano, etc.), I thought it was time to give people some more information about luminance and how to compare it to other frameworks.

Origin

luminance started as a Haskell package, extracted from a “3D engine” I had been working on for a while called quaazar. I came to the realization that I wasn’t using the Haskell garbage collector at all and that I could benefit from using a language without GC. Rust is a very famous language and well appreciated in the Haskell community, so I decided to jump in and learn Rust. I migrated luminance in a month or two. The mapping is described in this blog entry.

What is luminance for?

I’ve been writing 3D applications for a while and I always was frustrated by how OpenGL is badly designed. Let’s sum up the lack of design of OpenGL:

• weakly typed: OpenGL has types, but… it actually does not. GLint, GLuint or GLbitfield are all defined as aliases to primary and integral types (i.e. something like typedef float GLfloat). Try it with grep -Rn "typedef [a-zA-Z]* GLfloat" /usr/include/GL. This leads to the fact that framebuffers, textures, shader stages, shader program or even uniforms, etc. have the same type (GLuint, i.e. unsigned int). Thus, a function like glCompileShader expects a GLuint as argument, though you can pass a framebuffer, because it’s also represented as a GLuint – very bad for us. It’s better to consider that those are just untyped – :( – handles.
• runtime overhead: Because of the point above, functions cannot assume you’re passing a value of a the expected type – e.g. the example just above with glCompileShader and a framebuffer. That means OpenGL implementations have to check against all the values you’re passing as arguments to be sure they match the type. That’s basically several tests for each call of an OpenGL function. If the type doesn’t match, you’re screwed and see the next point.
• error handling: This is catastrophic. Because of the runtime overhead, almost all functions might set the error flag. You have to check the error flag with the glGetError function, adding a side-effect, preventing parallelism, etc.
• global state: OpenGL works on the concept of global mutation. You have a state, wrapped in a context, and each time you want to do something with the GPU, you have to change something in the context. Such a context is important; however, some mutations shouldn’t be required. For instance, when you want to change the value of an object or use a texture, OpenGL requires you to bind the object. If you forget to bind for the next object, the mutation will occurs on the first object. Side effects, side effects…

The goal of luminance is to fix most of those issues by providing a safe, stateless and elegant graphics framework. It should be as low-level as possible, but shouldn’t sacrifice runtime performances – CPU charge as well as memory bandwidth. That is why if you know how to program with OpenGL, you won’t feel lost when getting your feet wet with luminance.

Because of the many OpenGL versions and other technologies (among them, vulkan), luminance has an extra aim: abstract over the trending graphics API.

Types in luminance

In luminance, all graphics resources – and even concepts – have their own respective type. For instance, instead of GLuint for both shader programs and textures, luminance has Program and Texture. That ensures you don’t pass values with the wrong types.

Because of static warranties provided by compile-time, with such a scheme of strong-typing, the runtime shouldn’t have to check for type safety. Unfortunately, because luminance wraps over OpenGL in the luminance-gl backend, we can only add static warranties; we cannot remove the runtime overhead.

Error handling

luminance follows the Rust conventions and uses the famous Option and Result types to specify errors. You will never have to check against a global error flag, because this is just all wrong. Keep in mind, you have the try! macro in your Rust prelude; use it as often as possible!

Even though Rust needs to provide an exception handler – i.e. panics – there’s no such thing as exceptions in Rust. The try! macro is just syntactic sugar to:

match result {
  Ok(x) => x,
  Err(e) => return e
}

Stateless

luminance is stateless. That means you don’t have to bind an object to be able to use it. luminance takes care of that for you in a very simple way. To achieve this and keep performances running, it’s required to add a bit of high-level to the OpenGL API by leveraging how binds should happen.

Whatever the task you’re trying to reach, whatever computation or problem, it’s always better to gather / group the computation by batches. A good example of that is how magnetic hard drive disks work or your RAM. If you spread your data across the disk region (fragmented data) or across several non-contiguous addresses in your RAM, it will end up by unnecessary moves. The hard drive’s head will have to go all over the disk to gather the information, and it’s very likely you’ll destroy the RAM performance (and your CPU caches) if you don’t put the data in a contiguous area.

If you don’t group your OpenGL resources – for instances, you render 400 objects with shader A, 10 objects with shader B, then 20 objects with shader A, 32 objects with shader C, 349 objects with shader A and finally 439 objects with shader B, you’ll add more OpenGL calls to the equation – hence more global state mutations, and those are costly.

Instead of this:

1. 400 objects with shader A
2. 10 objects with shader B
3. 20 objects with shader A
4. 32 objects with shader C
5. 348 objects with shader A
6. 439 objects with shader B

luminance forces you to group your resources like this:

1. 400 + 20 + 348 objects with shader A
2. 10 + 439 objects with shader B
3. 32 objects with shader C

This is done via types called Pipeline, ShadingCommand and RenderCommand.

Pipelines

A Pipeline gathers shading commands under a Framebuffer. That means that all ShadingCommand embedded in the Pipeline will output to the embedded Framebuffer. Simple, yet powerful, because we can bind the framebuffer when executing the pipeline and don’t have to worry about framebuffer until the next execution of another Pipeline.

ShadingCommand

A ShadingCommand gathers render commands under a shader Program along with an update function. The update function is used to customize the Program by providing uniforms – i.e. Uniform. If you want to change a Programs Uniform once a frame – and only if the Program is only called once in the frame – it’s the right place to do it.

All RenderCommand embedded in the ShadingCommand will be rendered using the embedded shader Program. Like with the Pipeline, we don’t have to worry about binding: we just have to use the embedded shader program when executing the ShadingCommand, and we’ll bind another program the next time a ShadingCommand is ran!

RenderCommand

A RenderCommand gathers all the information required to render a Tessellation, that is:

• the blending equation, source and destination blending factors
• whether the depth test should be performed
• an update function to update the Program being in use – so that each object can have different properties used in the shader program
• a reference to the Tessellation to render
• the number of instances of the Tessellation to render
• the size of the rasterized points (if the Tessellation contains any)

What about shaders?

Shaders are written in… the backend’s expected format. For OpenGL, you’ll have to write GLSL. The backends automatically inserts the version pragma (#version 330 core for OpenGL 3.3 for instance). In the first place, I wanted to migrate cheddar, my Haskell shader EDSL. But… the sad part of the story is that Rust is – yet – unable to handle that kind of stuff correctly. I started to implement an EDSL for luminance with macros. Even though it was usable, the error handling is seriously terrible – macros shouldn’t be used for such an important purpose. Then some rustaceans pointed out I could implement a (rustc) compiler plugin. That enables the use of new constructs directly into Rust by extending its syntax. This is great.

However, with the hindsight, I will not do that. For a very simple reason. luminance is, currently, simple, stateless and most of all: it works! I released a PC demo in Köln, Germany using luminance and a demoscene graphics framework I’m working on:

pouët.net link

youtube capture

ion demoscene framework

While developping Céleri Rémoulade, I decided to bake the shaders directly into Rust – to get used to what I had wanted to build, i.e., a shader EDSL. So there’re a bunch of constant &'static str everywhere. Each time I wanted to make a fix to a shader, I had to leave the application, make the change, recompile, rerun… I’m not sure it’s a good thing. Interactive programming is a very good thing we can enjoy – yes, even in strongly typed languages ;).

I saw that gfx doesn’t have its own shader EDSL either and requires you to provide several shader implementations (one per backend). I don’t know; I think it’s not that bad if you only target a single backend (i.e. OpenGL 3.3 or Vulkan). Transpiling shaders is a thing, I’ve been told…

sneaking out…

Feel free to dig in the code of Céleri Rémoulade here. It’s demoscene code, so it had been rushed on before the release – read: it’s not as clean as I wanted it to be.

I’ll provide you with more information in the next weeks, but I prefer spending my spare time writing code than explaining what I’m gonna do – and missing the time to actually do it. ;)

Keep the vibe!

Porting a Haskell graphics framework to Rust (luminance)

I wanted to write that new article to discuss about something important I’ve been doing for several weeks. It’s actually been a month that I’ve been working on luminance, but not in the usual way. Yeah, I’ve put my Haskell experience aside to… port luminance into Rust! There are numerous reasons why I decided to jump in and I think it could be interesting for people to know about the differences I’ve been facing while porting the graphics library.

You said Rust?

Yeah, Rust. It’s a strong and static language aiming at system programming. Although it’s an imperative language, it has interesting functional conventions that caught my attention. Because I’m a haskeller and because Rust takes a lot from Haskell, learning it was a piece of cake, even though there are a few concepts I needed a few days to wrap my mind around. Having a strong C++11/14 experience, it wasn’t that hard though.

How does it compare to Haskell?

The first thing that amazed me is the fact that it’s actually not that different from Haskell! Rust has a powerful type system – not as good as Haskell’s but still – and uses immutability as a default semantic for bindings, which is great. For instance, the following is forbidden in Rust and would make rustc – the Rust compiler – freak out:

let a = "foo";
a = "bar"; // wrong; forbidden

Haskell works like that as well. However, you can introduce mutation with the mut keyword:

let mut a = "foo";
a = "bar"; // ok

Mutation should be used only when needed. In Haskell, we have the ST monad, used to introduce local mutation, or more drastically the IO monad. Under the wood, those two monads are actually almost the same type – with different warranties though.

Rust is strict by default while Haskell is lazy. That means that Rust doesn’t know the concept of memory suspensions, or thunks – even though you can create them by hand if you want to. Thus, some algorithms are easier to implement in Haskell thanks to laziness, but some others will destroy your memory if you’re not careful enough – that’s a very common problem in Haskell due to thunks piling up in your stack / heap / whatever as you do extensive lazy computations. While it’s possible to remove those thunks by optimizing a Haskell program – profiling, strictness, etc., Rust doesn’t have that problem because it gives you full access to the memory. And that’s a good thing if you need it. Rust exposes a lot of primitives to work with memory. In contrast with Haskell, it doesn’t have a garbage collector, so you have to handle memory on your own. Well, not really. Rust has several very interesting concepts to handle memory in a very nice way. For instance, objects’ memory is held by scopes – which have lifetimes. RAII is a very well known use of that concept and is important in Rust. You can glue code to your type that will be ran when an instance of that type dies, so that you can clean up memory and scarce resources.

Rust has the concept of lifetimes, used to give names to scopes and specify how long an object reference should live. This is very powerful yet a bit complex to understand in the first place.

I won’t go into comparing the two languages because it would require several articles and a lot of spare time I don’t really have. I’ll stick to what I’d like to tell you: the Rust implementation of luminance.

Porting luminance from Haskell to Rust

The first very interesting aspect of that port is the fact that it originated from a realization while refactoring some of my luminance Haskell code. Although it’s functional, stateless and type-safe, a typical use of luminance doesn’t really require laziness nor a garbage collector. And I don’t like using a tool – read language – like a bazooka. Haskell is the most powerful language ever in terms of abstraction and expressivity over speed ratio, but all of that power comes with an overhead. Even though you’ll find folks around stating that Haskell is pretty okay to code a video game, I think it will never compete with languages that are made to solve real time computations or reactive programming. And don’t get me wrong: I’m sure you can write a decent video game in Haskell – I qualify myself as a Haskeller and I’ve not been writing luminance just for the joy of writing it. However, the way I use Haskell with luminance shouldn’t require all the overhead – and profiling got me right, almost no GC was involved.

So… I looked into Rust and discovered and learned the language in only three days. I think it’s due to the fact that Rust, which is simpler than Haskell in terms of type system features and has almost everything taken from Haskell, is, to me, an imperative Haskell. It’s like having a Haskell minus a few abstraction tools – HKT (but they’ll come soon), GADTs, fundeps, kinds, constraints, etc. – plus a total control of what’s happening. And I like that. A lot. A fucking lot.

Porting luminance to Rust wasn’t hard as a Haskell codebase might map almost directly to Rust. I had to change a few things – for instance, Rust doesn’t have the concept of existential quantification as-is, which is used intensively in the Haskell version of luminance. But most Haskell modules map directly to their respective Rust modules. I changed the architecture of the files to have something clearer. I was working on loose coupling in Haskell for luminance. So I decided to directly introduce loose coupling into the Rust version. And it works like a charm.

So there are, currently, two packages available: luminance, which is the core API, exporting the whole general interface, and luminance-gl, an OpenGL 3.3 backend – though it will contain more backends as the development goes on. The idea is that you need both the dependencies to have access to luminance’s features.

I won’t say much today because I’m working on a demoscene production using luminance. I want it to be a proof that the framework is usable, works and acts as a first true example. Of course, the code will be open-source.

The documentation is not complete yet but I put some effort documenting almost everything. You’ll find both the packages here:

luminance-0.1.0

luminance-gl-0.1.0

I’ll write another article on how to use luminance as soon as possible!

Keep the vibe!

Existential quantification and GADT in luminance-0.8

luminance 0.8 and existential quantification

It’s been a while I haven’t released anything on my blog. I just wrote a few changes for the latest version of luminance, luminance-0.8.2 and I decided to write about it because I think those changes are interesting on a Haskell level.

The problem

If you haven’t read the changelog yet, I changed the createProgram function and the way it handles uniform interfaces. In luminance < 0.8, you were provided with as many functions as there are uniform kinds. Up to now, luminance supports two uniform kinds:

• simple uniforms;
  • uniform block (UBO).

So you had two rank-2 functions like forall a. (Uniform a) => Either String Natural -> UniformInterface m (U a) and forall a. (UniformBlock a) => String -> UniformInterface m (U (Region rw (UB a))) to map whichever uniforms you wanted to.

The issue with that is that it requires to break the interface of createProgram each time we want to add a new kind of uniform, and it’s also a pretty hard to read function signature!

So… how does luminance-0.8 solve that?

(Generalized) Algebraic data types, rank-2 and existential quantification

What is the only way we have to select uniforms? Names. Names can either be a String or a Natural for explicit semantics. We could encode such a name using an algebraic data type:

data UniformName
  = UniformName String
    | UniformSemantic Natural
      deriving (Eq,Show)

That’s a good start. Though, we still have the problem of choosing the kind of uniform because we still have several functions – one per kind. We could encode the kind of the uniform directly into the name. After all, when we ask for a uniform mapping through a name, we require to know the kind. So that kind of makes sense. Let’s change our UniformName type:

data UniformName :: * -> * where
  UniformName :: String -> UniformName a
    UniformSemantic :: Natural -> UniformName a
    UniformBlockName :: String -> UniformName (Region rw (UB a))

That’s neat, but with that definition, we won’t go anywhere, because we’re too polymorphic. Indeed, UniformName "foo" :: UniformName a can have any a. We need to put constraints on a. And that’s where GADTs come in so handy! We can hide the constraints in the constructors and bring them into scope when pattern matching. That’s a very neat feature of GADTs. So now, let’s add some constraints to our constructors:

data UniformName :: * -> * where
  UniformName :: (Uniform a) => String -> UniformName a
    UniformSemantic :: (Uniform a) => Natural -> UniformName a
    UniformBlockName :: (UniformBlock a) => String -> UniformName (Region rw (UB a))

Yes! Now, we can write a function that takes a UniformName a, pattern matches it and call the appropriate function regarding the infered shape of a!

However, how do we forward the error? In older version of luminance, we were using ProgramError and more especially two of its constructors: InactiveUniform and InactiveUniformBlock. We need to shrink that to a single InactiveUniform constructor and find a way to store our UniformName… But we can’t yet, because of the a parameter! So the idea is to hide it through existential quantification!

data SomeUniformName = forall a. SomeUniformName (UniformName a)

instance Eq SomeUniformName where
  -- …

instance Show SomeUniformName where
  -- …

And now we can store SomeUniformName in InactiveUniform. We won’t need to recover the type, we just need the constructor and the carried name. By pattern matching, we can recover both those information!

Conclusion

Feel free to have a look at the new createProgram function. As you will see, the type signature is easier to read and to work with! :)

Have fun, and keep the vibe!

OpenGL 3.2 support for luminance!

luminance-0.7

You can’t even imagine how hard it was to release luminance-0.7. I came accross several difficulties I had to spend a lot of time on but finally, here it is. I made a lot of changes for that very special release, and I have a lot to say about it!

Overview

As for all my projects, I always provide people with a changelog. The 0.7 release is a major release (read as: it was a major increment). I think it’s good to tell people what’s new, but it should be mandatory to warn them about what has changed so that they can directly jump to their code and spot the uses of the deprecated / changed interface.

Anyway, you’ll find patch, minor and major changes in luminance-0.7. I’ll describe them in order.

Patch changes

Internal architecture and debugging

A lot of code was reviewed internally. You don’t have to worry about that. However, there’s a new cool thing that was added internally. It could have been marked as a minor change but it’s not supposed to be used by common people – you can use it via a flag if you use cabal or stack though. It’s about debugging the OpenGL part used in luminance. You shouldn’t have to use it but it could be interesting if you spot a bug someday. Anyway, you can enable it with the flag debug-gl.

Uniform Block / Uniform Buffer Objects

The UBO system was buggy and was fixed. You might experience issue with them though. I spotted a bug and reported it – you can find the bug report here. That bug is not Haskell related and is related to the i915 Intel driver.

Minor changes

The minor changes were the most important part of luminance-0.7. luminance now officially supports OpenGL 3.2! When installing luminance, you default to the gl32 backend. You can select the backend you want with flags – gl45 and gl45-bindless-textures – but keep in mind you need the appropriate hardware to be able to use it. Because you need to use flags, you won’t be able to switch to the backend you want at runtime – that’s not the purpose of such a change though.

The performance gap should be minor between gl32 and gl45 but still. Basically, OpenGL 4.5 adds the support for DSA, which is very handy and less ill-designed that previous iterations of OpenGL. So a lot of code had to be rewritten to implement luminance’s stateless interface without breaking performance nor bring them down.

I might add support for other backends later on – like an OpenGL ES backend and WebGL one – but that won’t ship that soon though because I have a ton of work to do, and yet need to provide you with a concrete, beautiful, fast, appealing and eye-blowing demo with luminance! ;)

Feel free to test the gl32 backend and give me back feedback!

However, if I spent so much time on that 0.7 version, it’s because I had issue whilst writing the gl32 backend. Indeed, I spotted several bugs on my Intel HD card. This is my OpenGL version string for my Intel IGP card:

OpenGL core profile version string: 3.3 (Core Profile) Mesa 11.0.4

The architecture is Haswell. And on such a card (i915 linux driver) I’ve found two bugs while trying the gl32 backend with luminance-samples-0.7.

usampler2D

For unknown reason, the Texture sample failed on my Intel IGP but ran smoothly and flawlessly on my nVidia GPU. I spent a lot of time trying to figure out what I was missing, but eventually changed the sampler type – it’s now a sampler2D – and… it worked. I reported the issue to the intel dev team. So if you hit that error too, please leave a message here so that I can have more hindsight about that error and see what I can do.

Uniform block and vec3

This is a very nasty issue that kept me awoken for days trying to fix my code while it was a driver bug. It’s a big technical, so I’ll just leave a link to the bug tracker so that you can read it if you want to.

Breaking changes

Ok, let’s talk.

When creating a new shader stage, you now have to use the function createStage – instead of several functions like createVertexShader. That change is important so that I can add new shader types without changing the interface, and because some shader can fail to be created. For instance, on the gl32 backend, trying to build a tessellation shader will raise an error.

When a shader stage creation fails, the UnsupportedStage error is raised and holds the type of the stage that failed.

Finally, the interface for the cubemaps changed a bit – you don’t have access to width and height anymore, that was error-prone and useless; you’re stick to a size parameter.

I’d like to thank all people supporting me and luminance. I’ll be watching reactions to that major and important release as it will cover more people with cheaper and well-spread GPUs.

Happy hacking! :)

luminance, episode 0.6: UBO, SSBO, Stackage.

Up to now, luminance has been lacking two cool features: UBO and SSBO. Both are buffer-backed uniform techniques. That is, a way to pass uniforms to shader stages through buffers.

The latest version of luminance has one of the two features. UBO were added and SSBO will follow for the next version, I guess.

What is UBO?

UBO stands for Uniform Bbuffer Object. Basically, it enables you to create uniform blocks in GLSL in feed them with buffers. Instead of passing values directly to the uniform interface, you just write whatever values you want to to buffers, and then pass the buffer as a source for the uniform block.

Such a technique has a lot of advantages. Among them, you can pass a lot of values. It’s also cool when you want to pass values instances of a structure (in the GLSL source code). You can also use them to share uniforms between several shader programs as well as quickly change all the uniforms to use.

In luminance, you need several things. First thing first, you need… a buffer! More specifically, you need a buffer Region to store values in. However, you cannot use any kind of region. You have to use a region that can hold values that will be fetched from shaders. This is done with a type called UB a. A buffer of UB a can be used as UBO.

Let’s say you want to store colors in a buffer, so that you can use them in your fragment shader. We’ll want three colors to shade a triangle. We need to create the buffer and get the region:

colorBuffer :: Region RW (UB (V3 Float)) <- createBuffer (newRegion 3)

The explicit type is there so that GHC can infer the correct types for the Region. As you can see, nothing fancy, except that we just don’t want a Region RW (V3 Float but Region RW (UB (V3 Float)). Why RW?

Then, we’ll want to store colors in the buffer. Easy peasy:

writeWhole colorBuffer (map UB colors)

colors :: [V3 Float]
colors = [V3 1 0 0,V3 0 1 0,V3 0 0 1] -- red, green, blue

At this point, colorBuffer represents a GPU buffer that holds three colors: red, green and blue. The next part is to get the uniform interface. That part is experimental in terms of exposed interface, but the core idea will remain the same. You’re given a function to build UBO uniforms as you also have a function to build simple and plain uniforms in createProgram:

createProgram shaderList $ \uni uniBlock -> {- … -}

Don’t spend too much time reading the signature of that function. You just have to know that uni is a function that takes Either String Natural – either a uniform’s name or its integral semantic – and gives you mapped U in return and that uniBlock does the same thing, but for uniform blocks instead.

Here’s our vertex shader:

in vec2 co;
out vec4 vertexColor;

// This is the uniform block, called "Colors" and storing three colors
// as an array of three vec3 (RGB).
uniform Colors {
  vec3 colors[3];
};

void main() {
  gl_Position = vec4(co, 0., 1.);
  vertexColor = vec4(colors[gl_VertexID], 1.);
}"

So we want to get a U a mapped to that "Colors" uniform block. Easy!

(program,colorsU) <- createProgram shaderStages $ \_ uniBlock -> uniBlock "Colors"

And that’s all! The type of colorsU is U (Region rw (UB (V3 Float))). You can then gather colorBuffer and colorsU in a uniform interface to send colorBuffer to colorsU!

You can find the complete sample here.

Finally, you can augment the type you can use UB with by implementing the UniformBlock typeclass. You can derive the Generic typeclass and then use a default instance:

data MyType = {- … -} deriving (Generic)

instance UniformBlock MyTpe -- we’re good to go with buffer of MyType!

luminance, luminance-samples and Stackage

I added luminance and luminance-samples into Stackage. You can then find them in the nightly snapshots and the future LTS ones.

What’s next?

I plan to add stencil support for the framebuffer, because it’s missing and people might like it included. I will of course add support for *SSBO** as soon as I can. I also need to work on cheddar but that project is complex and I’m still stuck with design decisions.

Thanks for reading my and for your feedback. Have you great week!

luminance-0.5.1 and wavefront-0.4.0.1

It’s been a few days I haven’t talked about luminance. I’ve been working on it a lot those days along with wavefront. In order that you keep up to date, I’ll describe the changes I made in those packages you have a talk about the future directions of those packages.

I’ll also give a snippet you can use to load geometries with wavefront and adapt them to embed into luminance so that you can actually render them! A package might come up from that kind of snippet – luminance-wavefront? We’ll see that!

wavefront

This package has received several changes among two major increments and several fixes. In the first place, I removed some code from the interface that was useless and used only for test purposes. I removed the Ctxt object – it’s a type used by the internal lexer anyways, so you don’t have to know about it – and exposed a type called WavefrontOBJ. That type reprents the parsed Wavefront data and is the main type used by the library in the interface.

Then, I also removed most of the modules, because they’re re-exported by the main module – Codec.Wavefront. I think the documentation is pretty straight-forward, but you think something is missing, please shoot a PM or an email! ;)

On the bugs level, I fixed a few things. Among them, there was a nasty bug in the implementation of an internal recursive parser that caused the last wavefront statement to be silently ignored.

I’d also like to point out that I performed some benchmarks – I will provide the data later on with a heap profile and graphs – and I’m pretty astonished with the results! The parser/lexer is insanely fast! It only takes a few milliseconds (between 7ms and 8ms) to load 50k faces (a 2MB .obj file). The code is not yet optimized, so I guess the package could go even faster!

You can find the changelog here.

luminance

I made a lot of work on luminance lately. First, the V type – used to represent vertex components – is not anymore defined by luminance but by linear. You can find the type here. You’ll need the DataKinds extension to write types like V 3 Float.

That change is due to the fact linear is a mature library with a lot of interesting functions and types everyone might use when doing graphics. Its V type has several interesting instances – Storable, Ord, etc. – that are required in luminance. Because it’s not simple to build such V, luminance provides you with three functions to build the 1D, 2D and 3D versions – vec2, vec3 and vec4. Currently, that type is the only one you can use to build vertex components. I might add V2, V3 and V4 as well later.

An interesting change: the Uniform typeclass has a lot of new instances! Basically, all vector types from linear, their array version and the 4x4 floating matrix – M44 Float. You can find the list of all instances here.

A new function was added to the Graphics.Lumimance.Geometry module called nubDirect. That function performs in linear logarithmic time and is used to turn a direct representation of vertices into a pair of data used to represent indexed vertices. The new list of vertices stores only unique vertices and the list of integral values stores the indices. You can then use both the information to build indexed geometries – see createGeometry for further details.

The interface to transfer texels to textures has changed. It doesn’t depend on Foldable anymore but on Data.Vector.Storable.Vector. That change is due to the fact that the Foldable solution uses toList behind the hood, which causes bad performance for the simple reason that we send the list to the GPU through the FFI. It’s then more efficient to use a Storable version. Furthermore, th most known package for textures loading – JuicyPixels – already uses that type of Vector. So you just have to enjoy the new performance boost! ;)

About bugs… I fixed a few ones. First, the implementation of the Storable instance for (:.) had an error for sizeOf. The implementation must be lazy in its argument, and the old one was not, causing undefined crashes when using that type. The strictness was removed and now everything works just fine!

Two bugs that were also fixed: the indexed render and the render of geometries with several vertex components. Those bugs were easy to fix and now you won’t experience those issues anymore.

Interfacing luminance with wavefront to render geometries from artists!

I thought it would be a hard task but I’m pretty proud of how easy it was to interface both the packages! The idea was to provide a function that would turn a WavefrontOBJ into a direct representation of luminance vertices. Here’s the function that implements such a conversion:

type Vtx = V 3 Float :. V 3 Float -- location :. normal

objToDirect :: WavefrontOBJ -> Maybe [Vtx]
objToDirect obj = traverse faceToVtx (toList faces)
  where
    locations = objLocations obj
    normals = objNormals obj
    faces = objFaces obj
    faceToVtx face = do
      let face' = elValue face
      vni <- faceNorIndex face'
      v <- locations !? (faceLocIndex face' - 1)
      vn <- normals !? (vni - 1)
      let loc = vec3 (locX v) (locY v) (locZ v)
          nor = vec3 (norX vn) (norY vn) (norZ vn)
      pure (loc :. nor)

As you can see, that function is pure and will eventually turn a WavefrontOBJ into a list of Vtx. Vtx is our own vertex type, encoding the location and the normal of the vertex. You can add texture coordinates if you want to. The function fails if a face’s index has no normal associated with or if an index is out-of-bound.

And… and that’s all! You can already have your Geometry with that – direct one:

  x <- fmap (fmap objToDirect) (fromFile "./ubercool-mesh.obj")
  case x of
    Right (Just vertices) -> createGeometry vertices Nothing Triangle
    _ -> throwError {- whatever you need as error there -}

You want an indexed version? Well, you already have everything to do that:

  x <- fmap (fmap (nubDirect . objToDirect) (fromFile "./ubercool-mesh.obj")
  case x of
    Right (Just (vertices,indices)) -> createGeometry vertices (Just indices) Triangle
    _ -> throwError {- whatever you need as error there -}

Even though the nubDirect performs in a pretty good complexity, it takes time. Don’t be surprised to see the “loading” time longer then.

I might package those snippets and helpers around them into a luminance-wavefront package, but that’s not trivial as the vertex format should be free.

Future directions and thank you

I received a lot of warm feedback from people about what I do in the Haskell community, and I’m just amazed. I’d like to thank each and everyone of you for your support – I even got support from non-Haskellers!

What’s next then… Well, I need to add a few more textures to luminance – texture arrays are not supported yet, and the framebuffers have to be altered to support all kind of textures. I will also try to write a cheddar interpreter directly into luminance to dump the String type of shader stages and replace it with cheddar’s whatever will be. For the long terms, I’ll add UBO and SSBO to luminance, and… compatibility with older OpenGL versions.

Once again, thank you, and keep the vibe!

luminance-0.3 – Adding more texture kinds to the equation…

Unleashing the power of textures!

From luminance-0.1 to luminance-0.2 included, it was not possible to use any texture types different than two-dimensional textures. This blog entry tags the new release, luminance-0.3, which adds support for several kinds of texture.

A bit more dimensions

Texture1D, Texture2D and Texture3D are all part of the new release. The interface has changed – hence the breaking changes yield a major version increment – and I’ll explain how it has.

Basically, textures are now fully polymorphic and are constrained by a typeclass: Texture. That typeclass enables ad hoc polymorphism. It is then possible to add more texture types without having to change the interface, which is cool. Like everything else in luminance, you just have to ask the typesystem which kind of texture you want, and everything will be taken care of for you.

Basically, you have three functions to know:

• createTexture, which is used to create a new texture ;
• uploadSub, used to upload texels to a subpart of the texture ;
• fillSub, used to fill – clear – a subpart of the texture with a given value.

All those functions work on (Texture t) => t, so it will work with all kinds of texture.

Cubemaps

Cubemaps are also included. They work like other textures but add the concept of faces. Feel free to dig in the documentation for further details.

What’s next?

I need to find a way to wrap texture arrays, which are very nice and useful for layered rendering. After that, I’ll try to expose the change to the framebuffers so that we can create framebuffers with cubemaps or that kind of cool feature.

In the waiting, have a good a week!

Luminance – ASAP

We’re almost there!

luminance, the Haskell graphics framework I’ve been working on for a month and a half, will be released very soon as 0.1 on hackage. I’m still working actively on several parts of it, especially the embedded documentation, wikis and main interface.

Keep in mind that the internal design is 80% done, but the end-user interface might change a lot in the future. Because I’m a demoscener, I’ll be using luminance for the next months to release a demoscene production in Germany and provide you with a nice feedback about how usable it is, so that I can make it more mature later on.

What to expect?

Currently, luminance works. You can create buffers, shaders, framebuffers, textures and blend the whole thing to create nice (animated) images. Everything is strongly and (almost) dependently typed, so that you have an extra type safety.

As I was developing the interface, I also wrote a new package that will be released on hackage as well: luminance-samples. As you might have guessed, that package contains several executables you can launch to test luminance. Those are just features sets. There’s an Hello, World! executable, a depth test executable, a blending one, a texture one, and so on and so forth. I’ll refactor them to make the code cleaner, but you should have a look to see what it entails to use luminance! ;)

I’ll be very open-minded about what you guys think of luminance once it gets released. Even though I’ve started writing it for my own purposes, I clearly understand that a lot of people are interested in that project. I’ve been contacted by the developers of waylandmonad to explain them the choices I made with luminance so that they can do the same thing when migrating xmonad from the Xorg technology to Wayland. If I can help in any ways, even if it’s not about luminance directly, don’t hesitate then contact me!

I can’t give you a 0.1 release milestone yet, but you should be able to install it from hackage and stackage very soon! I’ll write an article when it gets released, I promise.

In the waiting, keep the vibe. Happy hacking around!

Contravariance and luminance to add safety to uniforms

It’s been a few days I haven’t posted about luminance. I’m on holidays, thus I can’t be as involved in the development of the graphics framework as I’m used to on a daily basis. Although I’ve been producing less in the past few days, I’ve been actively thinking about something very important: uniform.

What people usually do

Uniforms are a way to pass data to shaders. I won’t talk about uniform blocks nor uniform buffers – I’ll make a dedicated post for that purpose. The common OpenGL uniform flow is the following:

1. you ask OpenGL to retrieve the location of a GLSL uniform through the function glGetUniformLocation, or you can use an explicit location if you want to handle the semantics on your own ;
2. you use that location, the identifier of your shader program and send the actual values with the proper glProgramUniform.

You typically don’t retrieve the location each time you need to send values to the GPU – you only retrieve them once, while initializing.

The first thing to make uniforms more elegant and safer is to provide a typeclass to provide a shared interface. Instead of using several functions for each type of uniform – glProgramUniform1i for Int32, glProgramUniform1f for Float and so on – we can just provide a function that will call the right OpenGL function for the type:

class Uniform a where
  sendUniform :: GLuint -> GLint -> a -> IO ()

instance Uniform Int32 where
  sendUniform = glProgramUniform1i

instance Uniform Float where
  sendUniform = glProgramUniform1f

-- and so on…

That’s the first step, and I think everyone should do that. However, that way of doing has several drawbacks:

• it still relies on side-effects; that is, we can call sendUniform pretty much everywhere ;
• imagine we have a shader program that requires several uniforms to be passed each time we draw something; what happens if we forget to call a sendUniform? If we haven’t sent the uniform yet, we might have an undefined behavior. If we already have, we will override all future draws with that value, which is very wrong… ;
• with that way of representing uniforms, we have a very imperative interface; we can have a more composable and pure approach than that, hence enabling us to gain in power and flexibility.

What luminance used to do

In my luminance package, I used to represent uniforms as values.

newtype U a = U { runU :: a -> IO () }

We can then alter the Uniform typeclass to make it simpler:

class Uniform a where
  toU :: GLuint -> GLint -> U a

instance Uniform Int32 where
  toU prog l = U $ glProgramUniform1i prog l

instance Uniform Float where
  toU prog l = U $ glProgramUniform1f prog l

We also have a pure interface now. I used to provide another type, Uniformed, to be able to send uniforms without exposing IO, and an operator to accumulate uniforms settings, (@=):

newtype Uniformed a = Uniformed { runUniformed :: IO a } deriving (Applicative,Functor,Monad)

(@=) :: U a -> a -> Uniformed ()
U f @= a = Uniformed $ f a

Pretty simple.

The new uniform interface

The problem with that is that we still have the completion problem and the side-effects, because we just wrap them without adding anything special – Uniformed is isomorphic to IO. We have no way to create a type and ensure that all uniforms have been sent down to the GPU…

Contravariance to save us!

If you’re an advanced Haskell programmer, you might have noticed something very interesting about our U type. It’s contravariant in its argument. What’s cool about that is that we could then create new uniform types – new U – by contramapping over those types! That means we can enrich the scope of the hardcoded Uniform instances, because the single way we have to get a U is to use Uniform.toU. With contravariance, we can – in theory – extend those types to all types.

Sounds handy eh? First thing first, contravariant functor. A contravariant functor is a functor that flips the direction of the morphism:

class Contravariant f where
  contramap :: (a -> b) -> f b -> f a
  (>$) :: b -> f b -> f a

contramap is the contravariant version of fmap and (>$) is the contravariant version of (<$). If you’re not used to contravariance or if it’s the first time you see such a type signature, it might seem confusing or even magic. Well, that’s the mathematic magic in the place! But you’ll see just below that there’s no magic no trick in the implementation.

Because U is contravariant in its argument, we can define a Contravariant instance:

instance Contravariant U where
  contramap f u = U $ runU u . f

As you can see, nothing tricky here. We just apply the (a -> b) function on the input of the resulting U a so that we can pass it to u, and we just runU the whole thing.

A few friends of mine – not Haskeller though – told me things like “That’s just theory bullshit, no one needs to know what a contravariant thingy stuff is!”. Well, here’s an example:

newtype Color = Color {
    colorName :: String
  , colorValue :: (Float,Float,Float,Float)
  }

Even though we have an instance of Uniform for (Float,Float,Float,Float), there will never be an instance of Uniform for Color, so we can’t have a U Color… Or can we?

uColor = contramap colorValue float4U

The type of uColor is… U Color! That works because contravariance enabled us to adapt the Color structure so that we end up on (Float,Float,Float,Float). The contravariance property is then a very great ally in such situations!

More contravariance

We can even dig in deeper! Something cool would be to do the same thing, but for several fields. Imagine a mouse:

data Mouse = Mouse {
    mouseX :: Float
  , mouseY :: Float
  }

We’d like to find a cool way to have U Mouse, so that we can send the mouse cursor to shaders. We’d like to contramap over mouseX and mouseY. A bit like with Functor + Applicative:

getMouseX :: IO Float
getMouseY :: IO Float

getMouse :: IO Mouse
getMouse = Mouse <$> getMouseX <*> getMouseY

We could have the same thing for contravariance… And guess what. That exists, and that’s called divisible contravariant functors! A Divisible contravariant functor is the exact contravariant version of Applicative!

class (Contravariant f) => Divisible f where
  divide :: (a -> (b,c)) -> f b -> f c -> f a
  conquer :: f a

divide is the contravariant version of (<*>) and conquer is the contravariant version of pure. You know that pure’s type is a -> f a, which is isomorphic to (() -> a) -> f a. Take the contravariant version of (() -> a) -> f a, you end up with (a -> ()) -> f a. (a -> ()) is isomorphic to (), so we can simplify the whole thing to f a. Here you have conquer. Thank you to Edward Kmett for helping me understand that!

Let’s see how we can implement Divisible for U!

instance Divisible U where
  divide f p q = U $ \a -> do
    let (b,c) = f a
    runU p b
    runU q c
  conquer = U . const $ pure ()

And now let’s use it to get a U Mouse!

let uMouse = divide (\(Mouse mx my) -> (mx,my)) mouseXU mouseYU

And here we have uMouse :: U Mouse! As you can see, if you have several uniforms – for each fields of the type, you can divide your type and map all fields to the uniforms by applying several times divide.

The current implementation is almost the one shown here. There’s also a Decidable instance, but I won’t talk about that for now.

The cool thing about that is that I can lose the Uniformed monadic type and rely only on U. Thanks to the Divisible typeclass, we have completion, and we can’t override future uniforms then!

---

I hope you’ve learnt something cool and useful through this. Keep in mind that category abstractions are powerful and are useful in some contexts.

Keep hacking around, keep being curious. A Haskeller never stops learning! And that’s what so cool about Haskell! Keep the vibe, and see you another luminance post soon!

Luminance – what was that alignment stuff already?

Yesterday, I released a new article about how I implement vertex arrays in luminance. In that article, I told you that the memory was packed with alignment set to 1.

Well, I’ve changed my mind. Some people pointed out that the good thing to do for most GPU is to align on 32-bit. That is, 4 bytes. The alignment should be 4 bytes, then, not 1.

There might be an issue with that. If you store a structure with attributes which sizes are not a multiple of 4 bytes, it’s likely you need to add padding.

However, I just reviewed my code, and found this:

instance (GPU a,KnownNat n,Storable a) => Vertex (V n a) where
instance (Vertex a,Vertex b) => Vertex (a :. b) where

Those are the single instances for Vertex. That means you can only use V and (:.) to build up vertices. Look at the V instance. You’ll find a GPU typeclass constraint. Let’s look at its definition and instances:

class GPU a where
  glType :: Proxy a -> GLenum

instance GPU Float where
  glType _ = GL_FLOAT

instance GPU Int32 where
  glType _ = GL_INT

instance GPU Word32 where
  glType _ = GL_UNSIGNED_INT

Woah. How did I forget that?! Let me translate those information to you. That means we can only have 32-bit vertex component! So the memory inside vertex buffers will always be aligned on 4 bytes! No need to worry about padding then!

The first implication is the fact you won’t be able to use Word16, for instance. You’ll need to stick to the three types that have a GPU instance.

Note: that doesn’t prevent us from adding Double later on, because a Double is a 64-bit type, which is a multiple of 4 bytes!

That’s all I have for today. I’m working on something very exciting linked to render batching. I’ll talk about that when it’s cooked. ;)

Keep the vibe; keep building awesome things, and as always, thank you for reading me!

Luminance – Vertex Arrays

I’ve been up working on vertex arrays in my work-in-progress graphics framework, luminance, for several days. I’m a bit slow, because I’ve been through a very hard breakup and have been struggling to recover and focus. But here I am!

So, what’s new?

OpenGL allows programmers to send vertices to the GPU through what is called a vertex array. Vertex specification is performed through several functions, operating on several objects. You need, for instance, a vertex buffer object, an index buffer object and a vertex array object. The vertex buffer stores the vertices data.

Teapot

For instance, you could imagine a teapot as a set of vertices. Those vertices have several attributes. We could use, for instance, a position, a normal and a bone index. The vertex buffer would be responsible of storing those positions, normals and bone indices. There’re two ways to store them:

1. interleaved arrays ;
2. deinterleaved arrays.

I’ll explain those later on. The index buffer stores integral numbers – mainly set to unsigned int – that index the vertices, so that we can connect them and create lines, triangles or more complex shapes.

Finally, the vertex array object is a state object that stores links to the two buffers and makes a connection between pointers in the buffer and attribute indices. Once everything is set up, we might only use the vertex array object. The exception is when we need to change the geometry of an object. We need to access the vertex buffer and the index buffer and upload new data. However, for now, that feature is disabled so that the buffers are not exposed to the programmer. If people think that feature should be implemented, I’ll create specialized code for that very purpose.

Interleaved and deinterleaved arrays

Interleaved arrays might be the most simple to picture, because you use such arrays every day when programming. Let’s imagine you have the following type in Haskell:

data Vertex = Vertex {
    vertPos    :: X
  , vertNor    :: Y
  , vertBoneID :: Z
  } deriving (Eq,Show)

Now, the teapot would have several vertices. Approximately, let’s state the teapot has five vertices – yeah, ugly teapot. We can represent such vertices in an interleaved array by simply recording them in a list or an array:

Interleaved

As you can see, the attributes are interleaved in memory, and the whole pattern is cycling. That’s the common way to represent an array of struct in a lot of languages, and it’s very natural for a machine to do things like that.

The deinterleaved version is:

Deinterleaved

As you can see, with deinterleaved arrays, all attributes are extracted and grouped. If you want to access the third vertex, you need to read the third X, the third Y and the third Z.

Both the methods have advantages and drawbacks. The cool thing about deinterleaved arrays is that we can copy huge regions of typed memory at once whilst we cannot with interleaved arrays. However, interleaved arrays store continuous structures, so writing and reading a structure back might be faster.

An important point to keep in mind: because we plan to pass those arrays to OpenGL, there’s no alignment restriction on the structure. That is, everything is packed, and we’ll have to pass extra information to OpenGL to tell it how to advance in memory to correctly build vertices back.

Generalized tuple

I think I haven’t told you yet. I have a cool type in luminance: the (:.) type. No, you don’t have to know how to pronounce that. I like to call it the gtuple type, because it’s a generalized tuple. You can encode (a,b), (a,b,c) and all kind of tuples with (:.). You can even encode single-typed infinite tuple! – a very special kind of list, indeed.

data a :. b = a :. b

infixr 6 :.

-- a :. b is isomorphic to (a,b)
-- a :. b :. c is isomorphic to (a,b,c)

newtype Fix f = Fix (f (Fix f)) -- from Control.Monad.Fix
type Inf a = Fix ((:.) a) -- infinite tuple!

Pretty simple, but way more powerful than the regular, monomorphic tuples. As you can see, (:.) is a right-associative. That means that a :. b :. c = a :. (b :. c).

That type will be heavily used in luminance, thus you should get your fet wet with it. There’s actually nothing much to know about it. It’s a Functor. I might add other features to it later on.

The Storable trick

The cool thing about (:.) is that we can provide a Storable instance for packed memory, as OpenGL requires it. Currently, the Storable instance is implemented like this:

instance (Storable a,Storable b) => Storable (a :. b) where
  sizeOf (a :. b) = sizeOf a + sizeOf b
  alignment _ = 1 -- packed data
  peek p = do
    a <- peek $ castPtr p
    b <- peek . castPtr $ p `plusPtr` sizeOf (undefined :: a)
    pure $ a :. b
  poke p (a :. b) = do
    poke (castPtr p) a
    poke (castPtr $ p `plusPtr` sizeOf (undefined :: a)) b

As you can see, the alignment is set to 1 to express the fact the memory is packed. The peek and poke functions use the size of the head of the tuple to advance the pointer so that we effectively write the whole tuple in packed memory.

Then, let’s rewrite our Vertex type in terms of (:.) to see how it’s going on:

type Vertex = X :. Y :. Z

If X, Y and Z are in Storable, we can directly poke one of our Vertex into a luminance buffer! That is, directly into the GPU buffer!

Keep in mind that the Storable instance implements packed-memory uploads and reads, and won’t work with special kinds of buffers, like shader storage ones, which require specific memory alignment. To cover them, I’ll create specific typeclasses instances. No worries.

Creating a vertex array

Creating a vertex array is done through the function createVertexArray. I might change the name of that object – it’s ugly, right? Maybe Shape, or something cooler!

createVertexArray :: (Foldable f,MonadIO m,MonadResource m,Storable v,Traversable t,Vertex v)
                  => t v
                  -> f Word32
                  -> m VertexArray

As you can see, the type signature is highly polymorphic. t and f represent foldable structures storing the vertices and the indices. And that’s all. Nothing else to feed the function with! As you can see, there’s a typeclass constraint on v, the inner vertex type, Vertex. That constraint ensures the vertex type is representable on the OpenGL side and has a known vertex format.

Disclaimer: the Traversable constraint might be relaxed to be Foldable very soon.

Once tested, I’ll move all that code from the unstable branch to the master branch so that you guys can test it. :)

About OpenGL…

I eventually came to the realization that I needed to inform you about the OpenGL prerequisites. Because I want the framework to be as modern and well-designed as possible, you’ll need… OpenGL 4.5. The latest version, indeed. You might also need an extension, ARB_bindless_texture. That would enable the framework to pass textures to shader in a very stateless way, which is our objective!

I’ll let you know what I decide about that. I don’t want to use an extension that is not implemented almost everywhere.

What’s next?

Well, tests! I need to be sure everything is correctly done on the GPU side, especially the vertex format specification. I’m pretty confident though.

Once the vertex arrays are tested, I’ll start defining a render interface as stateless as I can. As always, I’ll keep you informed!

Luminance – framebuffers and textures

I’m happily suprised that so many Haskell people follow luminance! First thing first, let’s tell you about how it grows.

Well, pretty quickly! There’s – yet – no method to make actual renders, because I’m still working on how to implement some stuff (I’ll detail that below), but it’s going toward the right direction!

Framebuffers

Something that is almost done is the framebuffer part. The main idea of framebuffers – in OpenGL – is supporting offscreen renders, so that we can render to several framebuffers and combine them in several fancy ways. Framebuffers are often bound textures, used to pass the rendered information around, especially to shaders, or to get the pixels through texture reads CPU-side.

The thing is… OpenGL’s framebuffers are tedious. You can have incomplete framebuffers if you don’t attach textures with the right format, or to the wrong attachment point. That’s why the framebuffer layer of luminance is there to solve that.

In luminance, a Framebuffer rw c d is a framebuffer with two formats. A color format, c, and a depth format, d. If c = (), then no color will be recorded. If d = (), then no depth will be recorded. That enables the use of color-only or depth-only renders, which are often optimized by GPU. It also includes a rw type variable, which has the same role as for Buffer. That is, you can have read-only, write-only or read-write framebuffers.

And of course, all those features – having a write-only depth-only framebuffer for instance – are set through… types! And that’s what is so cool about how things are handled in luminance. You just tell it what you want, and it’ll create the required state and manage it for you GPU-side.

Textures

The format types are used to know which textures to create and how to attach them internally. The textures are hidden from the interface so that you can’t mess with them. I still need to find a way to provide some kind of access to the information they hold, in order to use them in shaders for instance. I’d love to provide some kind of monoidal properties between framebuffers – to mimick gloss Monoid instance for its Picture type, basically.

You can create textures, of course, by using the createTexture w h mipmaps function. w is the width, h the height of the texture. mipmaps is the number of mipmaps you want for the texture.

You can then upload texels to the texture through several functions. The basic form is uploadWhole tex autolvl texels. It takes a texture tex and the texels to upload to the whole texture region. It’s your responsibility to ensure that you pass the correct number of texels. The texels are represented with a polymorphic type. You’re not bound to any kind of textures. You can pass a list of texels, a Vector of texels, or whatever you want, as long as it’s Foldable.

It’s also possible to fill the whole texture with a single value. In OpenGL slang, such an operation is often called clearing – clearing a buffer, clearing a texture, clearing the back buffer, and so on. You can do that with fillWhole.

There’re two over functions to work with subparts of textures, but it’s not interesting for the purpose of that blog entry.

Pixel format

The cool thing is the fact I’ve unified pixel formats. Textures and framebuffers share the same pixel format type (Format t c). Currently, they’re all phantom types, but I might unify them further and use DataKinds to promote them to the type-level. A format has two type variables, t and c.

t is the underlying type. Currently, it can be either Int32, Word32 or Float. I might add support for Double as well later on.

c is the channel type. There’re basically five channel types:

• CR r, a red channel ;
• CRG r g, red and green channels ;
• CRGB r g b, red, green and blue channels ;
• CRGBA r g b a, red, green, blue and alpha channels ;
• CDepth d, a depth channel (special case of CR; for depths only).

The type variables r, g, b, a and d represent channel sizes. There’re currently three kind of channel sizes:

• C8, for 8-bit ;
• C16, for 16-bit ;
• C32, for 32-bit.

Then, Format Float (CR C32) is a red channel, 32-bit float – the OpenGL equivalent is R32F. Format Word32 (CRGB C8 C8 C16) is a RGB channel with red and green 8-bit unsigned integer channels and the blue one is a 16-bit unsigned integer channel.

Of course, if a pixel format doesn’t exist on the OpenGL part, you won’t be able to use it. Typeclasses are there to enforce the fact pixel format can be represented on the OpenGL side.

Next steps

Currently, I’m working hard on how to represent vertex formats. That’s not a trivial task, because we can send vertices to OpenGL as interleaved – or not – arrays. I’m trying to design something elegant and safe, and I’ll keep you informed when I finally get something. I’ll need to find an interface for the actual render command, and I should be able to release something we can actually use!

By the way, some people already tried it (Git HEAD), and that’s amazing! I’ve created the unstable branch so that I can push unstable things, and keep the master branch as clean as possible.

Keep the vibe, and have fun hacking around!

Introducing Luminance, a safer OpenGL API

A few weeks ago, I was writing Haskell lines for a project I had been working on for a very long time. That project was a 3D engine. There are several posts about it on my blog, feel free to check out.

The thing is… Times change. The more it passes, the more I become mature in what I do in the Haskell community. I’m a demoscener, and I need to be productive. Writing a whole 3D engine for such a purpose is a good thing, but I was going round and round in circles, changing the whole architecture every now and then. I couldn’t make my mind and help it. So I decided to stop working on that, and move on.

If you are a Haskell developer, you might already know Edward Kmett. Each talk with him is always interesting and I always end up with new ideas and new knowledge. Sometimes, we talk about graphics, and sometimes, he tells me that writing a 3D engine from scratch and release it to the community is not a very good move.

I’ve been thinking about that, and in the end, I agree with Edward. There’re two reasons making such a project hard and not interesting for a community:

1. a good “3D engine” is a specialized one – for FPS games, for simulations, for sport games, for animation, etc. If we know what the player will do, we can optimize a lot of stuff, and put less details into not-important part of the visuals. For instance, some games don’t really care about skies, so they can use simple skyboxes with nice textures to bring a nice touch of atmosphere, without destroying performance. In a game like a flight simulator, skyboxes have to be avoided to go with other techniques to provide a correct experience to players. Even though an engine could provide both techniques, apply that problem to almost everything – i.e. space partitionning for instance – and you end up with a nightmare to code ;
2. an engine can be a very bloated piece of software – because of point 1. It’s very hard to keep an engine up to date regarding technologies, and make every one happy, especially if the engine targets a large audience of people – i.e. hackage.

Point 2 might be strange to you, but that’s often the case. Building a flexible 3D engine is a very hard and non-trivial task. Because of point 1, you utterly need to restrict things in order to get the required level of performance or design. There are people out there – especially in the demoscene world – who can build up 3D engines quickly. But keep in mind those engines are limited to demoscene applications, and enhancing them to support something else is not a trivial task. In the end, you might end up with a lot of bloated code you’ll eventually zap later on to build something different for another purpose – eh, demoscene is about going dirty, right?! ;)

Basics

So… Let’s go back to the basics. In order to include everyone, we need to provide something that everyone can download, install, learn and use. Something like OpenGL. For Haskell, I highly recommend using gl. It’s built against the gl.xml file – released by Khronos. If you need sound, you can use the complementary library I wrote, using the same name convention, al.

The problem with that is the fact that OpenGL is a low-level API. Especially for new comers or people who need to get things done quickly. The part that bothers – wait, no, annoys – me the most is the fact that OpenGL is a very old library which was designed two decades ago. And we suffer from that. A lot.

OpenGL is a stateful graphics library. That means it maintains a state, a context, in order to work properly. Maintaining a context or state is a legit need, don’t get it twisted. However, if the design of the API doesn’t fit such a way of dealing with the state, we come accross a lot of problems. Is there one programmer who hasn’t experienced black screens yet? I don’t think so.

The OpenGL’s API exposes a lot of functions that perform side-effects. Because OpenGL is weakly typed – almost all objects you can create in OpenGL share the same GL(u)int type, which is very wrong – you might end up doing nasty things. Worse, it uses an internal binding system to select the objects you want to operate on. For instance, if you want to upload data to a texture object, you need to bind the texture before calling the texture upload function. If you don’t, well, that’s bad for you. There’s no way to verify code safety at compile-time.

You’re not convinced yet? OpenGL doesn’t tell you directly how to change things on the GPU side. For instance, do you think you have to bind your vertex buffer before performing a render, or is it sufficient to bind the vertex array object only? All those questions don’t have direct answers, and you’ll need to dig in several wikis and forums to get your answers – the answer to that question is “Just bind the VAO, pal.”

What can we do about it?

Several attempts to enhance that safety have come up. The first thing we have to do is to wrap all OpenGL object types into proper types. For instance, we need several types for Texture and Framebuffer.

Then, we need a way to ensure that we cannot call a function if the context is not setup for. There are a few ways to do that. For instance, indexed monads can be a good start. However, I tried that, and I can tell you it’s way too complicated. You end up with very long types that make things barely unreadable. See this and this for excerpts.

Luminance

In my desperate quest of providing a safer OpenGL’s API, I decided to create a library from scratch called luminance. That library is not really an OpenGL safe wrapper, but it’s very close to being that.

luminance provides the same objects than OpenGL does, but via a safer way to create, access and use them. It’s an effort for providing safe abstractions without destroying performance down and suited for graphics applications. It’s not a 3D engine. It’s a rendering framework. There’s no light, asset managers or that kind of features. It’s just a tiny and simple powerful API.

Example

luminance is still a huge work in progress. However, I can already show an example. The following example opens a window but doesn’t render anything. Instead, it creates a buffer on the GPU and perform several simple operations onto it.

-- Several imports.
import Control.Monad.IO.Class ( MonadIO(..) )
import Control.Monad.Trans.Resource -- from the resourcet package
import Data.Foldable ( traverse_ )
import Graphics.Luminance.Buffer
import Graphics.Luminance.RW
import Graphics.UI.GLFW -- from the GLFW-b package
import Prelude hiding ( init ) -- clash with GLFW-b’s init function

windowW,windowH :: Int
windowW = 800
windowH = 600

windowTitle :: String
windowTitle = "Test"

main :: IO ()
main = do
  init
  -- Initiate the OpenGL context with GLFW.
  windowHint (WindowHint'Resizable False)
  windowHint (WindowHint'ContextVersionMajor 3)
  windowHint (WindowHint'ContextVersionMinor 3)
  windowHint (WindowHint'OpenGLForwardCompat False)
  windowHint (WindowHint'OpenGLProfile OpenGLProfile'Core)
  window <- createWindow windowW windowH windowTitle Nothing Nothing
  makeContextCurrent window
  -- Run our application, which needs a (MonadIO m,MonadResource m) => m
  -- we traverse_ so that we just terminate if we’ve failed to create the
  -- window.
  traverse_ (runResourceT . app) window
  terminate

-- GPU regions. For this example, we’ll just create two regions. One of floats
-- and the other of ints. We’re using read/write (RW) regions so that we can
-- send values to the GPU and read them back.
data MyRegions = MyRegions {
    floats :: Region RW Float
  , ints   :: Region RW Int
  }

-- Our logic.
app :: (MonadIO m,MonadResource m) => Window -> m ()
app window = do
  -- We create a new buffer on the GPU, getting back regions of typed data
  -- inside of it. For that purpose, we provide a monadic type used to build
  -- regions through the 'newRegion' function.
  region <- createBuffer $
    MyRegions
      <$> newRegion 10
      <*> newRegion 5
  clear (floats region) pi -- clear the floats region with pi
  clear (ints region) 10 -- clear the ints region with 10
  readWhole (floats region) >>= liftIO . print -- print the floats as an array
  readWhole (ints region) >>= liftIO . print -- print the ints as an array
  floats region `writeAt` 7 $ 42 -- write 42 at index=7 in the floats region
  floats region @? 7 >>= traverse_ (liftIO . print) -- safe getter (Maybe)
  floats region @! 7 >>= liftIO . print -- unsafe getter
  readWhole (floats region) >>= liftIO . print -- print the floats as an array

Those read/write regions could also have been made read-only or write-only. For such regions, some functions can’t be called, and trying to do so will make your compiler angry and throw errors at you.

Up to now, the buffers are created persistently and coherently. That might cause issues with OpenGL synchronization, but I’ll wait for benchmarks before changing that part. If benchmarking spots performance bottlenecks, I’ll introduce more buffers and regions to deal with special cases.

luminance doesn’t force you to use a specific windowing library. You can then embed it into any kind of host libraries.

What’s to come?

luminance is very young. At the moment of writing this article, it’s only 26 commits old. I just wanted to present it so that people know it exists will be released as soon as possible. The idea is to provide a library that, if you use it, won’t create black screens because of framebuffers incorrectness or buffers issues. It’ll ease debugging OpenGL applications and prevent from making nasty mistakes.

I’ll keep posting about luminance as I get new features implemented.

As always, keep the vibe, and happy hacking!