luminance designs

luminance-0.7.0 was released a few days ago and I decided it was time to explain exactly what luminance is and what were the design choices I made. After a very interesting talk with nical about other rust graphics frameworks (e.g. gfx, glium, vulkano, etc.), I thought it was time to give people some more information about luminance and how to compare it to other frameworks.

Origin

luminance started as a Haskell package, extracted from a “3D engine” I had been working on for a while called quaazar. I came to the realization that I wasn’t using the Haskell garbage collector at all and that I could benefit from using a language without GC. Rust is a very famous language and well appreciated in the Haskell community, so I decided to jump in and learn Rust. I migrated luminance in a month or two. The mapping is described in this blog entry.

What is luminance for?

I’ve been writing 3D applications for a while and I always was frustrated by how OpenGL is badly designed. Let’s sum up the lack of design of OpenGL:

• weakly typed: OpenGL has types, but… it actually does not. GLint, GLuint or GLbitfield are all defined as aliases to primary and integral types (i.e. something like typedef float GLfloat). Try it with grep -Rn "typedef [a-zA-Z]* GLfloat" /usr/include/GL. This leads to the fact that framebuffers, textures, shader stages, shader program or even uniforms, etc. have the same type (GLuint, i.e. unsigned int). Thus, a function like glCompileShader expects a GLuint as argument, though you can pass a framebuffer, because it’s also represented as a GLuint – very bad for us. It’s better to consider that those are just untyped – :( – handles.
• runtime overhead: Because of the point above, functions cannot assume you’re passing a value of a the expected type – e.g. the example just above with glCompileShader and a framebuffer. That means OpenGL implementations have to check against all the values you’re passing as arguments to be sure they match the type. That’s basically several tests for each call of an OpenGL function. If the type doesn’t match, you’re screwed and see the next point.
• error handling: This is catastrophic. Because of the runtime overhead, almost all functions might set the error flag. You have to check the error flag with the glGetError function, adding a side-effect, preventing parallelism, etc.
• global state: OpenGL works on the concept of global mutation. You have a state, wrapped in a context, and each time you want to do something with the GPU, you have to change something in the context. Such a context is important; however, some mutations shouldn’t be required. For instance, when you want to change the value of an object or use a texture, OpenGL requires you to bind the object. If you forget to bind for the next object, the mutation will occurs on the first object. Side effects, side effects…

The goal of luminance is to fix most of those issues by providing a safe, stateless and elegant graphics framework. It should be as low-level as possible, but shouldn’t sacrifice runtime performances – CPU charge as well as memory bandwidth. That is why if you know how to program with OpenGL, you won’t feel lost when getting your feet wet with luminance.

Because of the many OpenGL versions and other technologies (among them, vulkan), luminance has an extra aim: abstract over the trending graphics API.

Types in luminance

In luminance, all graphics resources – and even concepts – have their own respective type. For instance, instead of GLuint for both shader programs and textures, luminance has Program and Texture. That ensures you don’t pass values with the wrong types.

Because of static warranties provided by compile-time, with such a scheme of strong-typing, the runtime shouldn’t have to check for type safety. Unfortunately, because luminance wraps over OpenGL in the luminance-gl backend, we can only add static warranties; we cannot remove the runtime overhead.

Error handling

luminance follows the Rust conventions and uses the famous Option and Result types to specify errors. You will never have to check against a global error flag, because this is just all wrong. Keep in mind, you have the try! macro in your Rust prelude; use it as often as possible!

Even though Rust needs to provide an exception handler – i.e. panics – there’s no such thing as exceptions in Rust. The try! macro is just syntactic sugar to:

match result {
  Ok(x) => x,
  Err(e) => return e
}

Stateless

luminance is stateless. That means you don’t have to bind an object to be able to use it. luminance takes care of that for you in a very simple way. To achieve this and keep performances running, it’s required to add a bit of high-level to the OpenGL API by leveraging how binds should happen.

Whatever the task you’re trying to reach, whatever computation or problem, it’s always better to gather / group the computation by batches. A good example of that is how magnetic hard drive disks work or your RAM. If you spread your data across the disk region (fragmented data) or across several non-contiguous addresses in your RAM, it will end up by unnecessary moves. The hard drive’s head will have to go all over the disk to gather the information, and it’s very likely you’ll destroy the RAM performance (and your CPU caches) if you don’t put the data in a contiguous area.

If you don’t group your OpenGL resources – for instances, you render 400 objects with shader A, 10 objects with shader B, then 20 objects with shader A, 32 objects with shader C, 349 objects with shader A and finally 439 objects with shader B, you’ll add more OpenGL calls to the equation – hence more global state mutations, and those are costly.

Instead of this:

1. 400 objects with shader A
2. 10 objects with shader B
3. 20 objects with shader A
4. 32 objects with shader C
5. 348 objects with shader A
6. 439 objects with shader B

luminance forces you to group your resources like this:

1. 400 + 20 + 348 objects with shader A
2. 10 + 439 objects with shader B
3. 32 objects with shader C

This is done via types called Pipeline, ShadingCommand and RenderCommand.

Pipelines

A Pipeline gathers shading commands under a Framebuffer. That means that all ShadingCommand embedded in the Pipeline will output to the embedded Framebuffer. Simple, yet powerful, because we can bind the framebuffer when executing the pipeline and don’t have to worry about framebuffer until the next execution of another Pipeline.

ShadingCommand

A ShadingCommand gathers render commands under a shader Program along with an update function. The update function is used to customize the Program by providing uniforms – i.e. Uniform. If you want to change a Programs Uniform once a frame – and only if the Program is only called once in the frame – it’s the right place to do it.

All RenderCommand embedded in the ShadingCommand will be rendered using the embedded shader Program. Like with the Pipeline, we don’t have to worry about binding: we just have to use the embedded shader program when executing the ShadingCommand, and we’ll bind another program the next time a ShadingCommand is ran!

RenderCommand

A RenderCommand gathers all the information required to render a Tessellation, that is:

• the blending equation, source and destination blending factors
• whether the depth test should be performed
• an update function to update the Program being in use – so that each object can have different properties used in the shader program
• a reference to the Tessellation to render
• the number of instances of the Tessellation to render
• the size of the rasterized points (if the Tessellation contains any)

What about shaders?

Shaders are written in… the backend’s expected format. For OpenGL, you’ll have to write GLSL. The backends automatically inserts the version pragma (#version 330 core for OpenGL 3.3 for instance). In the first place, I wanted to migrate cheddar, my Haskell shader EDSL. But… the sad part of the story is that Rust is – yet – unable to handle that kind of stuff correctly. I started to implement an EDSL for luminance with macros. Even though it was usable, the error handling is seriously terrible – macros shouldn’t be used for such an important purpose. Then some rustaceans pointed out I could implement a (rustc) compiler plugin. That enables the use of new constructs directly into Rust by extending its syntax. This is great.

However, with the hindsight, I will not do that. For a very simple reason. luminance is, currently, simple, stateless and most of all: it works! I released a PC demo in Köln, Germany using luminance and a demoscene graphics framework I’m working on:

pouët.net link

youtube capture

ion demoscene framework

While developping Céleri Rémoulade, I decided to bake the shaders directly into Rust – to get used to what I had wanted to build, i.e., a shader EDSL. So there’re a bunch of constant &'static str everywhere. Each time I wanted to make a fix to a shader, I had to leave the application, make the change, recompile, rerun… I’m not sure it’s a good thing. Interactive programming is a very good thing we can enjoy – yes, even in strongly typed languages ;).

I saw that gfx doesn’t have its own shader EDSL either and requires you to provide several shader implementations (one per backend). I don’t know; I think it’s not that bad if you only target a single backend (i.e. OpenGL 3.3 or Vulkan). Transpiling shaders is a thing, I’ve been told…

sneaking out…

Feel free to dig in the code of Céleri Rémoulade here. It’s demoscene code, so it had been rushed on before the release – read: it’s not as clean as I wanted it to be.

I’ll provide you with more information in the next weeks, but I prefer spending my spare time writing code than explaining what I’m gonna do – and missing the time to actually do it. ;)

Keep the vibe!

Porting a Haskell graphics framework to Rust (luminance)

I wanted to write that new article to discuss about something important I’ve been doing for several weeks. It’s actually been a month that I’ve been working on luminance, but not in the usual way. Yeah, I’ve put my Haskell experience aside to… port luminance into Rust! There are numerous reasons why I decided to jump in and I think it could be interesting for people to know about the differences I’ve been facing while porting the graphics library.

You said Rust?

Yeah, Rust. It’s a strong and static language aiming at system programming. Although it’s an imperative language, it has interesting functional conventions that caught my attention. Because I’m a haskeller and because Rust takes a lot from Haskell, learning it was a piece of cake, even though there are a few concepts I needed a few days to wrap my mind around. Having a strong C++11/14 experience, it wasn’t that hard though.

How does it compare to Haskell?

The first thing that amazed me is the fact that it’s actually not that different from Haskell! Rust has a powerful type system – not as good as Haskell’s but still – and uses immutability as a default semantic for bindings, which is great. For instance, the following is forbidden in Rust and would make rustc – the Rust compiler – freak out:

let a = "foo";
a = "bar"; // wrong; forbidden

Haskell works like that as well. However, you can introduce mutation with the mut keyword:

let mut a = "foo";
a = "bar"; // ok

Mutation should be used only when needed. In Haskell, we have the ST monad, used to introduce local mutation, or more drastically the IO monad. Under the wood, those two monads are actually almost the same type – with different warranties though.

Rust is strict by default while Haskell is lazy. That means that Rust doesn’t know the concept of memory suspensions, or thunks – even though you can create them by hand if you want to. Thus, some algorithms are easier to implement in Haskell thanks to laziness, but some others will destroy your memory if you’re not careful enough – that’s a very common problem in Haskell due to thunks piling up in your stack / heap / whatever as you do extensive lazy computations. While it’s possible to remove those thunks by optimizing a Haskell program – profiling, strictness, etc., Rust doesn’t have that problem because it gives you full access to the memory. And that’s a good thing if you need it. Rust exposes a lot of primitives to work with memory. In contrast with Haskell, it doesn’t have a garbage collector, so you have to handle memory on your own. Well, not really. Rust has several very interesting concepts to handle memory in a very nice way. For instance, objects’ memory is held by scopes – which have lifetimes. RAII is a very well known use of that concept and is important in Rust. You can glue code to your type that will be ran when an instance of that type dies, so that you can clean up memory and scarce resources.

Rust has the concept of lifetimes, used to give names to scopes and specify how long an object reference should live. This is very powerful yet a bit complex to understand in the first place.

I won’t go into comparing the two languages because it would require several articles and a lot of spare time I don’t really have. I’ll stick to what I’d like to tell you: the Rust implementation of luminance.

Porting luminance from Haskell to Rust

The first very interesting aspect of that port is the fact that it originated from a realization while refactoring some of my luminance Haskell code. Although it’s functional, stateless and type-safe, a typical use of luminance doesn’t really require laziness nor a garbage collector. And I don’t like using a tool – read language – like a bazooka. Haskell is the most powerful language ever in terms of abstraction and expressivity over speed ratio, but all of that power comes with an overhead. Even though you’ll find folks around stating that Haskell is pretty okay to code a video game, I think it will never compete with languages that are made to solve real time computations or reactive programming. And don’t get me wrong: I’m sure you can write a decent video game in Haskell – I qualify myself as a Haskeller and I’ve not been writing luminance just for the joy of writing it. However, the way I use Haskell with luminance shouldn’t require all the overhead – and profiling got me right, almost no GC was involved.

So… I looked into Rust and discovered and learned the language in only three days. I think it’s due to the fact that Rust, which is simpler than Haskell in terms of type system features and has almost everything taken from Haskell, is, to me, an imperative Haskell. It’s like having a Haskell minus a few abstraction tools – HKT (but they’ll come soon), GADTs, fundeps, kinds, constraints, etc. – plus a total control of what’s happening. And I like that. A lot. A fucking lot.

Porting luminance to Rust wasn’t hard as a Haskell codebase might map almost directly to Rust. I had to change a few things – for instance, Rust doesn’t have the concept of existential quantification as-is, which is used intensively in the Haskell version of luminance. But most Haskell modules map directly to their respective Rust modules. I changed the architecture of the files to have something clearer. I was working on loose coupling in Haskell for luminance. So I decided to directly introduce loose coupling into the Rust version. And it works like a charm.

So there are, currently, two packages available: luminance, which is the core API, exporting the whole general interface, and luminance-gl, an OpenGL 3.3 backend – though it will contain more backends as the development goes on. The idea is that you need both the dependencies to have access to luminance’s features.

I won’t say much today because I’m working on a demoscene production using luminance. I want it to be a proof that the framework is usable, works and acts as a first true example. Of course, the code will be open-source.

The documentation is not complete yet but I put some effort documenting almost everything. You’ll find both the packages here:

luminance-0.1.0

luminance-gl-0.1.0

I’ll write another article on how to use luminance as soon as possible!

Keep the vibe!

Pure API vs. IO-bound API for graphics frameworks

Hi! It’s been a while I haven’t posted something here. I haven’t been around for a few weeks and I’ve been missing writing here. I didn’t work that much on luminance or other projects, even though I still provided updates for stackage for my packages. I worked a bit on cheddar and I hope to be able to release it soon!

Although I didn’t add things nor write code at home, I thought a lot about graphics API designs.

Graphics API designs

Pure API

APIs such as lambdacube or GPipe are known to be graphics pure API. That means you don’t have use functions bound to IO. You use some kind of free monads or an AST to represent the code that will run on the target GPU. That pure design brings numerous advantages to us:

• it’s possible to write the GPU code in a declarative, free and elegant way;
• because of not being IO-bound, side-effects are reduced, which is good and improves the code safety;
• one can write GPU code and several interpreters or/and with different technologies, hence a loosely coupled graphics API;
• we could even imagine serializing GPU code to send it through network, store it in a file or whatever.

Those advantages are nice, but there are also drawbacks:

• acquiring resources might not be explicit anymore and none knows exactly when sparce resources will be loaded into memory; even though we could use warmup to solve that, warmup doesn’t help much with dynamic scene where some resources are loaded only if the context enables it (like the player being in a given area of the map, in a game, for instance);
• interfacing! people will have to learn how to use free monads, AST and all that stuff and how to interface it with their own code (FRP, render loops, etc.);
• performance; even though it should be okay, you’ll still hit a performance overhead by using pure graphics frameworks, because of internals mechanisms used to make it work.

So yeah, a pure graphics framework is very neat, but keep in mind there’s – so far – no proof it actually works, scales nor is usable for a decent high-performance for end-users. It’s the same dilemna as with Conal’s FRP: it’s nice, but we don’t really know whether it works “at large scale and in the real world”.

IO-bound API

Most of the API out there are IO-bound. OpenGL is a famous C API known to be one of the worst one on the level of side-effects and global mutations. Trust me, it’s truly wrong. However, the pure API as mentioned above are based on those impure IO-bound APIs. So we couldn’t do much without them.

There are side effects that are not that bad. For instance, in OpenGL, creating a new buffer is a side-effect: it requires that the CPU tell the GPU “Hey buddy, please create a buffer with that data, and please give me back a handle to it!”. Then the GPU would reply ”No problem pal, here’s your handle!”. This side-effect don’t harm anyone, so we shouldn’t worry about it too much.

However, there are nasty side-effects, like binding resources to the OpenGL context.

So what are advantages of IO-bound designs? Well:

• simple: indeed, you have handles and side-effects and you have a (too) fine control of the instruction flow;
• performance, because IO is the naked real-world monad;
• because IO is the high-order kinded type of any application (think of the main function), an IO API is simple to use in any kind of application;
• we can use (MonadIO m) => m to add extra flexibility and create interesting constraints.

And drawbacks:

• IO is very opaque and is not referentially transparent;
• IO is a dangerous type in which no one has no warranty about what’s going on;
• one can fuck up everything if they aren’t careful;
• safety is not enforced as in pure code.

What about luminance’s design?

Since the beginning, luminance has been an API built to be simple, type-safe and stateless.

Type-safe means that all objects you use belong to different type sets and cannot be mixed between each other implicitely – you have to use explicit functions to do so, and it has to be meaningful. For instance, you cannot create a buffer and state that the returned handle is a texture: the type system forbids it, while in OpenGL, almost all objects are in the GLuint set. It’s very confusing and you might end up passing a texture (GLuint) to a function expecting a framebuffer (GLuint). Pretty bad right?

Stateless means that luminance has no state. You don’t have a huge context you have to bind stuff against to make it work. Everything is stored in the objects you use directly and specific context operations are translated into a different workflow so that performance are not destroyed – for instance luminance uses batch rendering so that it performs smart resource bindings.

Lately, I’ve been thinking of all of that. Either turn the API pure or leave it the way it is. I started to implement a pure API using self-recursion. The idea is actually simple. Imagine this GPU type and the once function:

import Control.Arrow ( (***), (&&&) )
import Data.Function ( fix )

newtype GPU f a = GPU { runGPU :: f (a,GPU f a) }

instance (Functor f) => Functor (GPU f) where
  fmap f = GPU . fmap (f *** fmap f) . runGPU

instance (Applicative f) => Applicative (GPU f) where
  pure x = fix $ \g -> GPU $ pure (x,g)
  f <*> a = GPU $ (\(f',fn) (a',an) -> (f' a',fn <*> an)) <$> runGPU f <*> runGPU a

instance (Monad m) => Monad (GPU m) where
  return = pure
  x >>= f = GPU $ runGPU x >>= runGPU . f . fst

once :: (Applicative f) => f a -> GPU f a
once = GPU . fmap (id &&& pure)

We can then build pure values that will have a side-effect for resource acquisition and then hold the same value for ever with the once function:

let buffer = once createBufferIO

-- later, in IO
(_,buffer2) <- runGPU buffer

Above, the type of buffer and buffer2 is GPU IO Buffer. The first call runGPU buffer will execute the once function, calling the createBufferIO IO function and will return buffer2, which just stores a pure Buffer.

Self-recursion is great to implement local states like that and I advise having a look at the Auto type. You can also read my article on netwire, which uses self-recursion a lot.

However, I kinda think that a library should have well defined responsibilities, and building such a pure interface is not the responsibility of luminance because we can have type-safety and a stateless API without wrapping everything in that GPU type. I think that if we want such a pure type, we should add it later on, in a 3D engine or a dedicated framework – and that’s actually what I do for demoscene purposes in another, ultra secret project. ;)

The cool thing with luminance using MonadIO is the fact that it’s very easy to use in any kind of type that developpers want to use in their applications. I really don’t like frameworks which purpose is clearly not flow control that actually enforce flow control and wrapping types! I don’t want to end up with a Luminance type or LuminanceApplication type. It should be simple to use and seamless.

I actually start to think that I did too much about that pure API design idea. The most important part of luminance should be type-safety and statelessness. If one wants a pure API, then they should use FRP frameworks or write their own stuff – with free monads for instance, and it’s actually funny to build!

The next big steps for luminance will be to clean the uniform interfaces which is a bit inconsistent and unfriendly to use with render commands. I’ll let you know.

luminance-0.5.1 and wavefront-0.4.0.1

It’s been a few days I haven’t talked about luminance. I’ve been working on it a lot those days along with wavefront. In order that you keep up to date, I’ll describe the changes I made in those packages you have a talk about the future directions of those packages.

I’ll also give a snippet you can use to load geometries with wavefront and adapt them to embed into luminance so that you can actually render them! A package might come up from that kind of snippet – luminance-wavefront? We’ll see that!

wavefront

This package has received several changes among two major increments and several fixes. In the first place, I removed some code from the interface that was useless and used only for test purposes. I removed the Ctxt object – it’s a type used by the internal lexer anyways, so you don’t have to know about it – and exposed a type called WavefrontOBJ. That type reprents the parsed Wavefront data and is the main type used by the library in the interface.

Then, I also removed most of the modules, because they’re re-exported by the main module – Codec.Wavefront. I think the documentation is pretty straight-forward, but you think something is missing, please shoot a PM or an email! ;)

On the bugs level, I fixed a few things. Among them, there was a nasty bug in the implementation of an internal recursive parser that caused the last wavefront statement to be silently ignored.

I’d also like to point out that I performed some benchmarks – I will provide the data later on with a heap profile and graphs – and I’m pretty astonished with the results! The parser/lexer is insanely fast! It only takes a few milliseconds (between 7ms and 8ms) to load 50k faces (a 2MB .obj file). The code is not yet optimized, so I guess the package could go even faster!

You can find the changelog here.

luminance

I made a lot of work on luminance lately. First, the V type – used to represent vertex components – is not anymore defined by luminance but by linear. You can find the type here. You’ll need the DataKinds extension to write types like V 3 Float.

That change is due to the fact linear is a mature library with a lot of interesting functions and types everyone might use when doing graphics. Its V type has several interesting instances – Storable, Ord, etc. – that are required in luminance. Because it’s not simple to build such V, luminance provides you with three functions to build the 1D, 2D and 3D versions – vec2, vec3 and vec4. Currently, that type is the only one you can use to build vertex components. I might add V2, V3 and V4 as well later.

An interesting change: the Uniform typeclass has a lot of new instances! Basically, all vector types from linear, their array version and the 4x4 floating matrix – M44 Float. You can find the list of all instances here.

A new function was added to the Graphics.Lumimance.Geometry module called nubDirect. That function performs in linear logarithmic time and is used to turn a direct representation of vertices into a pair of data used to represent indexed vertices. The new list of vertices stores only unique vertices and the list of integral values stores the indices. You can then use both the information to build indexed geometries – see createGeometry for further details.

The interface to transfer texels to textures has changed. It doesn’t depend on Foldable anymore but on Data.Vector.Storable.Vector. That change is due to the fact that the Foldable solution uses toList behind the hood, which causes bad performance for the simple reason that we send the list to the GPU through the FFI. It’s then more efficient to use a Storable version. Furthermore, th most known package for textures loading – JuicyPixels – already uses that type of Vector. So you just have to enjoy the new performance boost! ;)

About bugs… I fixed a few ones. First, the implementation of the Storable instance for (:.) had an error for sizeOf. The implementation must be lazy in its argument, and the old one was not, causing undefined crashes when using that type. The strictness was removed and now everything works just fine!

Two bugs that were also fixed: the indexed render and the render of geometries with several vertex components. Those bugs were easy to fix and now you won’t experience those issues anymore.

Interfacing luminance with wavefront to render geometries from artists!

I thought it would be a hard task but I’m pretty proud of how easy it was to interface both the packages! The idea was to provide a function that would turn a WavefrontOBJ into a direct representation of luminance vertices. Here’s the function that implements such a conversion:

type Vtx = V 3 Float :. V 3 Float -- location :. normal

objToDirect :: WavefrontOBJ -> Maybe [Vtx]
objToDirect obj = traverse faceToVtx (toList faces)
  where
    locations = objLocations obj
    normals = objNormals obj
    faces = objFaces obj
    faceToVtx face = do
      let face' = elValue face
      vni <- faceNorIndex face'
      v <- locations !? (faceLocIndex face' - 1)
      vn <- normals !? (vni - 1)
      let loc = vec3 (locX v) (locY v) (locZ v)
          nor = vec3 (norX vn) (norY vn) (norZ vn)
      pure (loc :. nor)

As you can see, that function is pure and will eventually turn a WavefrontOBJ into a list of Vtx. Vtx is our own vertex type, encoding the location and the normal of the vertex. You can add texture coordinates if you want to. The function fails if a face’s index has no normal associated with or if an index is out-of-bound.

And… and that’s all! You can already have your Geometry with that – direct one:

  x <- fmap (fmap objToDirect) (fromFile "./ubercool-mesh.obj")
  case x of
    Right (Just vertices) -> createGeometry vertices Nothing Triangle
    _ -> throwError {- whatever you need as error there -}

You want an indexed version? Well, you already have everything to do that:

  x <- fmap (fmap (nubDirect . objToDirect) (fromFile "./ubercool-mesh.obj")
  case x of
    Right (Just (vertices,indices)) -> createGeometry vertices (Just indices) Triangle
    _ -> throwError {- whatever you need as error there -}

Even though the nubDirect performs in a pretty good complexity, it takes time. Don’t be surprised to see the “loading” time longer then.

I might package those snippets and helpers around them into a luminance-wavefront package, but that’s not trivial as the vertex format should be free.

Future directions and thank you

I received a lot of warm feedback from people about what I do in the Haskell community, and I’m just amazed. I’d like to thank each and everyone of you for your support – I even got support from non-Haskellers!

What’s next then… Well, I need to add a few more textures to luminance – texture arrays are not supported yet, and the framebuffers have to be altered to support all kind of textures. I will also try to write a cheddar interpreter directly into luminance to dump the String type of shader stages and replace it with cheddar’s whatever will be. For the long terms, I’ll add UBO and SSBO to luminance, and… compatibility with older OpenGL versions.

Once again, thank you, and keep the vibe!

Load geometries with wavefront-0.1!

I’ve been away from luminance for a few days because I wanted to enhance the graphics world of Haskell. luminance might be interesting, if you can’t use the art works of your artists, you won’t go any further for a real-world application. I decided that I to write a parser/lexer to load 3D geometries from files. The Wavefront OBJ is an old yet simple and efficient way of encoding such objects. It supports materials, surfaces and a lot of other cool stuff – I don’t cover them yet, though.

There’s a package out there to do that, but it hasn’t been updated since 2008 and has a lot of dependencies I don’t like (InfixApplicative, OpenGL, OpenGLCheck, graphicsFormats, Codec-Image-Devil, and so on…). I like to keep things ultra simple and lightweight. So here we go. wavefront.

Currently, my package only builds up a pure value you can do whatever you want with. Upload it to the GPU, modify it, pretty print it, perform some physics on it. Whatever you want. The interface is not frozen yet and I need to perform some benchmarks to see if I have to improve the performance – the lexer is very simple and naive, I’d be amazed if the performance were that good yet.

As always, feel free to contribute, and keep in mind that the package will move quickly along the performance axis.

luminance-0.3 – Adding more texture kinds to the equation…

Unleashing the power of textures!

From luminance-0.1 to luminance-0.2 included, it was not possible to use any texture types different than two-dimensional textures. This blog entry tags the new release, luminance-0.3, which adds support for several kinds of texture.

A bit more dimensions

Texture1D, Texture2D and Texture3D are all part of the new release. The interface has changed – hence the breaking changes yield a major version increment – and I’ll explain how it has.

Basically, textures are now fully polymorphic and are constrained by a typeclass: Texture. That typeclass enables ad hoc polymorphism. It is then possible to add more texture types without having to change the interface, which is cool. Like everything else in luminance, you just have to ask the typesystem which kind of texture you want, and everything will be taken care of for you.

Basically, you have three functions to know:

• createTexture, which is used to create a new texture ;
• uploadSub, used to upload texels to a subpart of the texture ;
• fillSub, used to fill – clear – a subpart of the texture with a given value.

All those functions work on (Texture t) => t, so it will work with all kinds of texture.

Cubemaps

Cubemaps are also included. They work like other textures but add the concept of faces. Feel free to dig in the documentation for further details.

What’s next?

I need to find a way to wrap texture arrays, which are very nice and useful for layered rendering. After that, I’ll try to expose the change to the framebuffers so that we can create framebuffers with cubemaps or that kind of cool feature.

In the waiting, have a good a week!

luminance first tutorial

Woah!

I’m very happy about people getting interested about my luminance graphics framework. I haven’t received use case feedback yet, but I’m pretty confident I will sooner or later.

In the waiting, I decided to write an embedded tutorial. It can be found here.

That tutorial explains all the basic types of luminance – not all though, you’ll have to dig in the documentation ;) – and describes how you should use it. I will try to add more documentation for each modules in order to end up with a very well documented piece of software!

Let’s sum up what you need

People on reddit complain – they are right to – about the fact the samples just “didn’t work. They actually did, but the errors were muted. I released luminance-0.1.1 to fix that issue. Now you’ll get the proper error messages.

The most common issue is when you try to run a sample without having the required hardware implementation. luminance requires OpenGL 4.5. On Linux, you might need to use primusrun or optirun if you have the Optimus technology. On Windows, I guess you have to allow the samples to run on the dedicated GPU. And on Mac OSX… I have no idea; primusrun / optirun, I’d go.

Anyways, I’d like to thank all people who have/will tried/try the package. As always, I’ll keep you informed about all the big steps I take about luminance. Keep the vibe!

luminance 0.1 released!

Here we are

luminance-0.1 was released yesterday night, along with luminance-samples-0.1! I’ll need to enhance the documentation and add directions so that people don’t feel too overwhelmed.

I’m also going to write a wiki to help people get their mind wrapped around luminance.

If you think something is missing; if you think something could be enhanced; or if you’ve found a bug, please, feel free to fill in an issue on the issues tracker.

Next big steps

I need to test the framework. I need a lot of tests. I’ll write a demoscene production with it so that I can give a good feedback to the community and prove that luminance can be used and works.

In the waiting, keep the vibe!

Luminance – ASAP

We’re almost there!

luminance, the Haskell graphics framework I’ve been working on for a month and a half, will be released very soon as 0.1 on hackage. I’m still working actively on several parts of it, especially the embedded documentation, wikis and main interface.

Keep in mind that the internal design is 80% done, but the end-user interface might change a lot in the future. Because I’m a demoscener, I’ll be using luminance for the next months to release a demoscene production in Germany and provide you with a nice feedback about how usable it is, so that I can make it more mature later on.

What to expect?

Currently, luminance works. You can create buffers, shaders, framebuffers, textures and blend the whole thing to create nice (animated) images. Everything is strongly and (almost) dependently typed, so that you have an extra type safety.

As I was developing the interface, I also wrote a new package that will be released on hackage as well: luminance-samples. As you might have guessed, that package contains several executables you can launch to test luminance. Those are just features sets. There’s an Hello, World! executable, a depth test executable, a blending one, a texture one, and so on and so forth. I’ll refactor them to make the code cleaner, but you should have a look to see what it entails to use luminance! ;)

I’ll be very open-minded about what you guys think of luminance once it gets released. Even though I’ve started writing it for my own purposes, I clearly understand that a lot of people are interested in that project. I’ve been contacted by the developers of waylandmonad to explain them the choices I made with luminance so that they can do the same thing when migrating xmonad from the Xorg technology to Wayland. If I can help in any ways, even if it’s not about luminance directly, don’t hesitate then contact me!

I can’t give you a 0.1 release milestone yet, but you should be able to install it from hackage and stackage very soon! I’ll write an article when it gets released, I promise.

In the waiting, keep the vibe. Happy hacking around!

Contravariance and luminance to add safety to uniforms

It’s been a few days I haven’t posted about luminance. I’m on holidays, thus I can’t be as involved in the development of the graphics framework as I’m used to on a daily basis. Although I’ve been producing less in the past few days, I’ve been actively thinking about something very important: uniform.

What people usually do

Uniforms are a way to pass data to shaders. I won’t talk about uniform blocks nor uniform buffers – I’ll make a dedicated post for that purpose. The common OpenGL uniform flow is the following:

1. you ask OpenGL to retrieve the location of a GLSL uniform through the function glGetUniformLocation, or you can use an explicit location if you want to handle the semantics on your own ;
2. you use that location, the identifier of your shader program and send the actual values with the proper glProgramUniform.

You typically don’t retrieve the location each time you need to send values to the GPU – you only retrieve them once, while initializing.

The first thing to make uniforms more elegant and safer is to provide a typeclass to provide a shared interface. Instead of using several functions for each type of uniform – glProgramUniform1i for Int32, glProgramUniform1f for Float and so on – we can just provide a function that will call the right OpenGL function for the type:

class Uniform a where
  sendUniform :: GLuint -> GLint -> a -> IO ()

instance Uniform Int32 where
  sendUniform = glProgramUniform1i

instance Uniform Float where
  sendUniform = glProgramUniform1f

-- and so on…

That’s the first step, and I think everyone should do that. However, that way of doing has several drawbacks:

• it still relies on side-effects; that is, we can call sendUniform pretty much everywhere ;
• imagine we have a shader program that requires several uniforms to be passed each time we draw something; what happens if we forget to call a sendUniform? If we haven’t sent the uniform yet, we might have an undefined behavior. If we already have, we will override all future draws with that value, which is very wrong… ;
• with that way of representing uniforms, we have a very imperative interface; we can have a more composable and pure approach than that, hence enabling us to gain in power and flexibility.

What luminance used to do

In my luminance package, I used to represent uniforms as values.

newtype U a = U { runU :: a -> IO () }

We can then alter the Uniform typeclass to make it simpler:

class Uniform a where
  toU :: GLuint -> GLint -> U a

instance Uniform Int32 where
  toU prog l = U $ glProgramUniform1i prog l

instance Uniform Float where
  toU prog l = U $ glProgramUniform1f prog l

We also have a pure interface now. I used to provide another type, Uniformed, to be able to send uniforms without exposing IO, and an operator to accumulate uniforms settings, (@=):

newtype Uniformed a = Uniformed { runUniformed :: IO a } deriving (Applicative,Functor,Monad)

(@=) :: U a -> a -> Uniformed ()
U f @= a = Uniformed $ f a

Pretty simple.

The new uniform interface

The problem with that is that we still have the completion problem and the side-effects, because we just wrap them without adding anything special – Uniformed is isomorphic to IO. We have no way to create a type and ensure that all uniforms have been sent down to the GPU…

Contravariance to save us!

If you’re an advanced Haskell programmer, you might have noticed something very interesting about our U type. It’s contravariant in its argument. What’s cool about that is that we could then create new uniform types – new U – by contramapping over those types! That means we can enrich the scope of the hardcoded Uniform instances, because the single way we have to get a U is to use Uniform.toU. With contravariance, we can – in theory – extend those types to all types.

Sounds handy eh? First thing first, contravariant functor. A contravariant functor is a functor that flips the direction of the morphism:

class Contravariant f where
  contramap :: (a -> b) -> f b -> f a
  (>$) :: b -> f b -> f a

contramap is the contravariant version of fmap and (>$) is the contravariant version of (<$). If you’re not used to contravariance or if it’s the first time you see such a type signature, it might seem confusing or even magic. Well, that’s the mathematic magic in the place! But you’ll see just below that there’s no magic no trick in the implementation.

Because U is contravariant in its argument, we can define a Contravariant instance:

instance Contravariant U where
  contramap f u = U $ runU u . f

As you can see, nothing tricky here. We just apply the (a -> b) function on the input of the resulting U a so that we can pass it to u, and we just runU the whole thing.

A few friends of mine – not Haskeller though – told me things like “That’s just theory bullshit, no one needs to know what a contravariant thingy stuff is!”. Well, here’s an example:

newtype Color = Color {
    colorName :: String
  , colorValue :: (Float,Float,Float,Float)
  }

Even though we have an instance of Uniform for (Float,Float,Float,Float), there will never be an instance of Uniform for Color, so we can’t have a U Color… Or can we?

uColor = contramap colorValue float4U

The type of uColor is… U Color! That works because contravariance enabled us to adapt the Color structure so that we end up on (Float,Float,Float,Float). The contravariance property is then a very great ally in such situations!

More contravariance

We can even dig in deeper! Something cool would be to do the same thing, but for several fields. Imagine a mouse:

data Mouse = Mouse {
    mouseX :: Float
  , mouseY :: Float
  }

We’d like to find a cool way to have U Mouse, so that we can send the mouse cursor to shaders. We’d like to contramap over mouseX and mouseY. A bit like with Functor + Applicative:

getMouseX :: IO Float
getMouseY :: IO Float

getMouse :: IO Mouse
getMouse = Mouse <$> getMouseX <*> getMouseY

We could have the same thing for contravariance… And guess what. That exists, and that’s called divisible contravariant functors! A Divisible contravariant functor is the exact contravariant version of Applicative!

class (Contravariant f) => Divisible f where
  divide :: (a -> (b,c)) -> f b -> f c -> f a
  conquer :: f a

divide is the contravariant version of (<*>) and conquer is the contravariant version of pure. You know that pure’s type is a -> f a, which is isomorphic to (() -> a) -> f a. Take the contravariant version of (() -> a) -> f a, you end up with (a -> ()) -> f a. (a -> ()) is isomorphic to (), so we can simplify the whole thing to f a. Here you have conquer. Thank you to Edward Kmett for helping me understand that!

Let’s see how we can implement Divisible for U!

instance Divisible U where
  divide f p q = U $ \a -> do
    let (b,c) = f a
    runU p b
    runU q c
  conquer = U . const $ pure ()

And now let’s use it to get a U Mouse!

let uMouse = divide (\(Mouse mx my) -> (mx,my)) mouseXU mouseYU

And here we have uMouse :: U Mouse! As you can see, if you have several uniforms – for each fields of the type, you can divide your type and map all fields to the uniforms by applying several times divide.

The current implementation is almost the one shown here. There’s also a Decidable instance, but I won’t talk about that for now.

The cool thing about that is that I can lose the Uniformed monadic type and rely only on U. Thanks to the Divisible typeclass, we have completion, and we can’t override future uniforms then!

---

I hope you’ve learnt something cool and useful through this. Keep in mind that category abstractions are powerful and are useful in some contexts.

Keep hacking around, keep being curious. A Haskeller never stops learning! And that’s what so cool about Haskell! Keep the vibe, and see you another luminance post soon!

Luminance – Vertex Arrays

I’ve been up working on vertex arrays in my work-in-progress graphics framework, luminance, for several days. I’m a bit slow, because I’ve been through a very hard breakup and have been struggling to recover and focus. But here I am!

So, what’s new?

OpenGL allows programmers to send vertices to the GPU through what is called a vertex array. Vertex specification is performed through several functions, operating on several objects. You need, for instance, a vertex buffer object, an index buffer object and a vertex array object. The vertex buffer stores the vertices data.

Teapot

For instance, you could imagine a teapot as a set of vertices. Those vertices have several attributes. We could use, for instance, a position, a normal and a bone index. The vertex buffer would be responsible of storing those positions, normals and bone indices. There’re two ways to store them:

1. interleaved arrays ;
2. deinterleaved arrays.

I’ll explain those later on. The index buffer stores integral numbers – mainly set to unsigned int – that index the vertices, so that we can connect them and create lines, triangles or more complex shapes.

Finally, the vertex array object is a state object that stores links to the two buffers and makes a connection between pointers in the buffer and attribute indices. Once everything is set up, we might only use the vertex array object. The exception is when we need to change the geometry of an object. We need to access the vertex buffer and the index buffer and upload new data. However, for now, that feature is disabled so that the buffers are not exposed to the programmer. If people think that feature should be implemented, I’ll create specialized code for that very purpose.

Interleaved and deinterleaved arrays

Interleaved arrays might be the most simple to picture, because you use such arrays every day when programming. Let’s imagine you have the following type in Haskell:

data Vertex = Vertex {
    vertPos    :: X
  , vertNor    :: Y
  , vertBoneID :: Z
  } deriving (Eq,Show)

Now, the teapot would have several vertices. Approximately, let’s state the teapot has five vertices – yeah, ugly teapot. We can represent such vertices in an interleaved array by simply recording them in a list or an array:

Interleaved

As you can see, the attributes are interleaved in memory, and the whole pattern is cycling. That’s the common way to represent an array of struct in a lot of languages, and it’s very natural for a machine to do things like that.

The deinterleaved version is:

Deinterleaved

As you can see, with deinterleaved arrays, all attributes are extracted and grouped. If you want to access the third vertex, you need to read the third X, the third Y and the third Z.

Both the methods have advantages and drawbacks. The cool thing about deinterleaved arrays is that we can copy huge regions of typed memory at once whilst we cannot with interleaved arrays. However, interleaved arrays store continuous structures, so writing and reading a structure back might be faster.

An important point to keep in mind: because we plan to pass those arrays to OpenGL, there’s no alignment restriction on the structure. That is, everything is packed, and we’ll have to pass extra information to OpenGL to tell it how to advance in memory to correctly build vertices back.

Generalized tuple

I think I haven’t told you yet. I have a cool type in luminance: the (:.) type. No, you don’t have to know how to pronounce that. I like to call it the gtuple type, because it’s a generalized tuple. You can encode (a,b), (a,b,c) and all kind of tuples with (:.). You can even encode single-typed infinite tuple! – a very special kind of list, indeed.

data a :. b = a :. b

infixr 6 :.

-- a :. b is isomorphic to (a,b)
-- a :. b :. c is isomorphic to (a,b,c)

newtype Fix f = Fix (f (Fix f)) -- from Control.Monad.Fix
type Inf a = Fix ((:.) a) -- infinite tuple!

Pretty simple, but way more powerful than the regular, monomorphic tuples. As you can see, (:.) is a right-associative. That means that a :. b :. c = a :. (b :. c).

That type will be heavily used in luminance, thus you should get your fet wet with it. There’s actually nothing much to know about it. It’s a Functor. I might add other features to it later on.

The Storable trick

The cool thing about (:.) is that we can provide a Storable instance for packed memory, as OpenGL requires it. Currently, the Storable instance is implemented like this:

instance (Storable a,Storable b) => Storable (a :. b) where
  sizeOf (a :. b) = sizeOf a + sizeOf b
  alignment _ = 1 -- packed data
  peek p = do
    a <- peek $ castPtr p
    b <- peek . castPtr $ p `plusPtr` sizeOf (undefined :: a)
    pure $ a :. b
  poke p (a :. b) = do
    poke (castPtr p) a
    poke (castPtr $ p `plusPtr` sizeOf (undefined :: a)) b

As you can see, the alignment is set to 1 to express the fact the memory is packed. The peek and poke functions use the size of the head of the tuple to advance the pointer so that we effectively write the whole tuple in packed memory.

Then, let’s rewrite our Vertex type in terms of (:.) to see how it’s going on:

type Vertex = X :. Y :. Z

If X, Y and Z are in Storable, we can directly poke one of our Vertex into a luminance buffer! That is, directly into the GPU buffer!

Keep in mind that the Storable instance implements packed-memory uploads and reads, and won’t work with special kinds of buffers, like shader storage ones, which require specific memory alignment. To cover them, I’ll create specific typeclasses instances. No worries.

Creating a vertex array

Creating a vertex array is done through the function createVertexArray. I might change the name of that object – it’s ugly, right? Maybe Shape, or something cooler!

createVertexArray :: (Foldable f,MonadIO m,MonadResource m,Storable v,Traversable t,Vertex v)
                  => t v
                  -> f Word32
                  -> m VertexArray

As you can see, the type signature is highly polymorphic. t and f represent foldable structures storing the vertices and the indices. And that’s all. Nothing else to feed the function with! As you can see, there’s a typeclass constraint on v, the inner vertex type, Vertex. That constraint ensures the vertex type is representable on the OpenGL side and has a known vertex format.

Disclaimer: the Traversable constraint might be relaxed to be Foldable very soon.

Once tested, I’ll move all that code from the unstable branch to the master branch so that you guys can test it. :)

About OpenGL…

I eventually came to the realization that I needed to inform you about the OpenGL prerequisites. Because I want the framework to be as modern and well-designed as possible, you’ll need… OpenGL 4.5. The latest version, indeed. You might also need an extension, ARB_bindless_texture. That would enable the framework to pass textures to shader in a very stateless way, which is our objective!

I’ll let you know what I decide about that. I don’t want to use an extension that is not implemented almost everywhere.

What’s next?

Well, tests! I need to be sure everything is correctly done on the GPU side, especially the vertex format specification. I’m pretty confident though.

Once the vertex arrays are tested, I’ll start defining a render interface as stateless as I can. As always, I’ll keep you informed!

Luminance – framebuffers and textures

I’m happily suprised that so many Haskell people follow luminance! First thing first, let’s tell you about how it grows.

Well, pretty quickly! There’s – yet – no method to make actual renders, because I’m still working on how to implement some stuff (I’ll detail that below), but it’s going toward the right direction!

Framebuffers

Something that is almost done is the framebuffer part. The main idea of framebuffers – in OpenGL – is supporting offscreen renders, so that we can render to several framebuffers and combine them in several fancy ways. Framebuffers are often bound textures, used to pass the rendered information around, especially to shaders, or to get the pixels through texture reads CPU-side.

The thing is… OpenGL’s framebuffers are tedious. You can have incomplete framebuffers if you don’t attach textures with the right format, or to the wrong attachment point. That’s why the framebuffer layer of luminance is there to solve that.

In luminance, a Framebuffer rw c d is a framebuffer with two formats. A color format, c, and a depth format, d. If c = (), then no color will be recorded. If d = (), then no depth will be recorded. That enables the use of color-only or depth-only renders, which are often optimized by GPU. It also includes a rw type variable, which has the same role as for Buffer. That is, you can have read-only, write-only or read-write framebuffers.

And of course, all those features – having a write-only depth-only framebuffer for instance – are set through… types! And that’s what is so cool about how things are handled in luminance. You just tell it what you want, and it’ll create the required state and manage it for you GPU-side.

Textures

The format types are used to know which textures to create and how to attach them internally. The textures are hidden from the interface so that you can’t mess with them. I still need to find a way to provide some kind of access to the information they hold, in order to use them in shaders for instance. I’d love to provide some kind of monoidal properties between framebuffers – to mimick gloss Monoid instance for its Picture type, basically.

You can create textures, of course, by using the createTexture w h mipmaps function. w is the width, h the height of the texture. mipmaps is the number of mipmaps you want for the texture.

You can then upload texels to the texture through several functions. The basic form is uploadWhole tex autolvl texels. It takes a texture tex and the texels to upload to the whole texture region. It’s your responsibility to ensure that you pass the correct number of texels. The texels are represented with a polymorphic type. You’re not bound to any kind of textures. You can pass a list of texels, a Vector of texels, or whatever you want, as long as it’s Foldable.

It’s also possible to fill the whole texture with a single value. In OpenGL slang, such an operation is often called clearing – clearing a buffer, clearing a texture, clearing the back buffer, and so on. You can do that with fillWhole.

There’re two over functions to work with subparts of textures, but it’s not interesting for the purpose of that blog entry.

Pixel format

The cool thing is the fact I’ve unified pixel formats. Textures and framebuffers share the same pixel format type (Format t c). Currently, they’re all phantom types, but I might unify them further and use DataKinds to promote them to the type-level. A format has two type variables, t and c.

t is the underlying type. Currently, it can be either Int32, Word32 or Float. I might add support for Double as well later on.

c is the channel type. There’re basically five channel types:

• CR r, a red channel ;
• CRG r g, red and green channels ;
• CRGB r g b, red, green and blue channels ;
• CRGBA r g b a, red, green, blue and alpha channels ;
• CDepth d, a depth channel (special case of CR; for depths only).

The type variables r, g, b, a and d represent channel sizes. There’re currently three kind of channel sizes:

• C8, for 8-bit ;
• C16, for 16-bit ;
• C32, for 32-bit.

Then, Format Float (CR C32) is a red channel, 32-bit float – the OpenGL equivalent is R32F. Format Word32 (CRGB C8 C8 C16) is a RGB channel with red and green 8-bit unsigned integer channels and the blue one is a 16-bit unsigned integer channel.

Of course, if a pixel format doesn’t exist on the OpenGL part, you won’t be able to use it. Typeclasses are there to enforce the fact pixel format can be represented on the OpenGL side.

Next steps

Currently, I’m working hard on how to represent vertex formats. That’s not a trivial task, because we can send vertices to OpenGL as interleaved – or not – arrays. I’m trying to design something elegant and safe, and I’ll keep you informed when I finally get something. I’ll need to find an interface for the actual render command, and I should be able to release something we can actually use!

By the way, some people already tried it (Git HEAD), and that’s amazing! I’ve created the unstable branch so that I can push unstable things, and keep the master branch as clean as possible.

Keep the vibe, and have fun hacking around!

Introducing Luminance, a safer OpenGL API

A few weeks ago, I was writing Haskell lines for a project I had been working on for a very long time. That project was a 3D engine. There are several posts about it on my blog, feel free to check out.

The thing is… Times change. The more it passes, the more I become mature in what I do in the Haskell community. I’m a demoscener, and I need to be productive. Writing a whole 3D engine for such a purpose is a good thing, but I was going round and round in circles, changing the whole architecture every now and then. I couldn’t make my mind and help it. So I decided to stop working on that, and move on.

If you are a Haskell developer, you might already know Edward Kmett. Each talk with him is always interesting and I always end up with new ideas and new knowledge. Sometimes, we talk about graphics, and sometimes, he tells me that writing a 3D engine from scratch and release it to the community is not a very good move.

I’ve been thinking about that, and in the end, I agree with Edward. There’re two reasons making such a project hard and not interesting for a community:

1. a good “3D engine” is a specialized one – for FPS games, for simulations, for sport games, for animation, etc. If we know what the player will do, we can optimize a lot of stuff, and put less details into not-important part of the visuals. For instance, some games don’t really care about skies, so they can use simple skyboxes with nice textures to bring a nice touch of atmosphere, without destroying performance. In a game like a flight simulator, skyboxes have to be avoided to go with other techniques to provide a correct experience to players. Even though an engine could provide both techniques, apply that problem to almost everything – i.e. space partitionning for instance – and you end up with a nightmare to code ;
2. an engine can be a very bloated piece of software – because of point 1. It’s very hard to keep an engine up to date regarding technologies, and make every one happy, especially if the engine targets a large audience of people – i.e. hackage.

Point 2 might be strange to you, but that’s often the case. Building a flexible 3D engine is a very hard and non-trivial task. Because of point 1, you utterly need to restrict things in order to get the required level of performance or design. There are people out there – especially in the demoscene world – who can build up 3D engines quickly. But keep in mind those engines are limited to demoscene applications, and enhancing them to support something else is not a trivial task. In the end, you might end up with a lot of bloated code you’ll eventually zap later on to build something different for another purpose – eh, demoscene is about going dirty, right?! ;)

Basics

So… Let’s go back to the basics. In order to include everyone, we need to provide something that everyone can download, install, learn and use. Something like OpenGL. For Haskell, I highly recommend using gl. It’s built against the gl.xml file – released by Khronos. If you need sound, you can use the complementary library I wrote, using the same name convention, al.

The problem with that is the fact that OpenGL is a low-level API. Especially for new comers or people who need to get things done quickly. The part that bothers – wait, no, annoys – me the most is the fact that OpenGL is a very old library which was designed two decades ago. And we suffer from that. A lot.

OpenGL is a stateful graphics library. That means it maintains a state, a context, in order to work properly. Maintaining a context or state is a legit need, don’t get it twisted. However, if the design of the API doesn’t fit such a way of dealing with the state, we come accross a lot of problems. Is there one programmer who hasn’t experienced black screens yet? I don’t think so.

The OpenGL’s API exposes a lot of functions that perform side-effects. Because OpenGL is weakly typed – almost all objects you can create in OpenGL share the same GL(u)int type, which is very wrong – you might end up doing nasty things. Worse, it uses an internal binding system to select the objects you want to operate on. For instance, if you want to upload data to a texture object, you need to bind the texture before calling the texture upload function. If you don’t, well, that’s bad for you. There’s no way to verify code safety at compile-time.

You’re not convinced yet? OpenGL doesn’t tell you directly how to change things on the GPU side. For instance, do you think you have to bind your vertex buffer before performing a render, or is it sufficient to bind the vertex array object only? All those questions don’t have direct answers, and you’ll need to dig in several wikis and forums to get your answers – the answer to that question is “Just bind the VAO, pal.”

What can we do about it?

Several attempts to enhance that safety have come up. The first thing we have to do is to wrap all OpenGL object types into proper types. For instance, we need several types for Texture and Framebuffer.

Then, we need a way to ensure that we cannot call a function if the context is not setup for. There are a few ways to do that. For instance, indexed monads can be a good start. However, I tried that, and I can tell you it’s way too complicated. You end up with very long types that make things barely unreadable. See this and this for excerpts.

Luminance

In my desperate quest of providing a safer OpenGL’s API, I decided to create a library from scratch called luminance. That library is not really an OpenGL safe wrapper, but it’s very close to being that.

luminance provides the same objects than OpenGL does, but via a safer way to create, access and use them. It’s an effort for providing safe abstractions without destroying performance down and suited for graphics applications. It’s not a 3D engine. It’s a rendering framework. There’s no light, asset managers or that kind of features. It’s just a tiny and simple powerful API.

Example

luminance is still a huge work in progress. However, I can already show an example. The following example opens a window but doesn’t render anything. Instead, it creates a buffer on the GPU and perform several simple operations onto it.

-- Several imports.
import Control.Monad.IO.Class ( MonadIO(..) )
import Control.Monad.Trans.Resource -- from the resourcet package
import Data.Foldable ( traverse_ )
import Graphics.Luminance.Buffer
import Graphics.Luminance.RW
import Graphics.UI.GLFW -- from the GLFW-b package
import Prelude hiding ( init ) -- clash with GLFW-b’s init function

windowW,windowH :: Int
windowW = 800
windowH = 600

windowTitle :: String
windowTitle = "Test"

main :: IO ()
main = do
  init
  -- Initiate the OpenGL context with GLFW.
  windowHint (WindowHint'Resizable False)
  windowHint (WindowHint'ContextVersionMajor 3)
  windowHint (WindowHint'ContextVersionMinor 3)
  windowHint (WindowHint'OpenGLForwardCompat False)
  windowHint (WindowHint'OpenGLProfile OpenGLProfile'Core)
  window <- createWindow windowW windowH windowTitle Nothing Nothing
  makeContextCurrent window
  -- Run our application, which needs a (MonadIO m,MonadResource m) => m
  -- we traverse_ so that we just terminate if we’ve failed to create the
  -- window.
  traverse_ (runResourceT . app) window
  terminate

-- GPU regions. For this example, we’ll just create two regions. One of floats
-- and the other of ints. We’re using read/write (RW) regions so that we can
-- send values to the GPU and read them back.
data MyRegions = MyRegions {
    floats :: Region RW Float
  , ints   :: Region RW Int
  }

-- Our logic.
app :: (MonadIO m,MonadResource m) => Window -> m ()
app window = do
  -- We create a new buffer on the GPU, getting back regions of typed data
  -- inside of it. For that purpose, we provide a monadic type used to build
  -- regions through the 'newRegion' function.
  region <- createBuffer $
    MyRegions
      <$> newRegion 10
      <*> newRegion 5
  clear (floats region) pi -- clear the floats region with pi
  clear (ints region) 10 -- clear the ints region with 10
  readWhole (floats region) >>= liftIO . print -- print the floats as an array
  readWhole (ints region) >>= liftIO . print -- print the ints as an array
  floats region `writeAt` 7 $ 42 -- write 42 at index=7 in the floats region
  floats region @? 7 >>= traverse_ (liftIO . print) -- safe getter (Maybe)
  floats region @! 7 >>= liftIO . print -- unsafe getter
  readWhole (floats region) >>= liftIO . print -- print the floats as an array

Those read/write regions could also have been made read-only or write-only. For such regions, some functions can’t be called, and trying to do so will make your compiler angry and throw errors at you.

Up to now, the buffers are created persistently and coherently. That might cause issues with OpenGL synchronization, but I’ll wait for benchmarks before changing that part. If benchmarking spots performance bottlenecks, I’ll introduce more buffers and regions to deal with special cases.

luminance doesn’t force you to use a specific windowing library. You can then embed it into any kind of host libraries.

What’s to come?

luminance is very young. At the moment of writing this article, it’s only 26 commits old. I just wanted to present it so that people know it exists will be released as soon as possible. The idea is to provide a library that, if you use it, won’t create black screens because of framebuffers incorrectness or buffers issues. It’ll ease debugging OpenGL applications and prevent from making nasty mistakes.

I’ll keep posting about luminance as I get new features implemented.

As always, keep the vibe, and happy hacking!