A Case for Open-Source GPU Drivers

Home / Linaro Blog / Community / A Case for Open-Source GPU Drivers

For some time after I signed on as the graphics working group tech lead at Linaro, upon meeting someone new, the first thing after “nice to meet you” was invariably, “When do we get free drivers?”.  Ideologically, this seems like an automatic “great idea”.  After all, Linux is all about free (definitely in terms of software, mostly in terms of beer :-), open-source software and the ability of the user to decide what (or what not) to run on his or her hardware.  Drivers, however, are a tricky subject.  As much as I would like to believe that software is software, my experience with a number of driver teams at various GPU and computer companies tells me that things are not always so simple.

What’s the big deal?  There are a couple of things really.  First, GPUs tend to be really different beasts from other devices.  There are huge variations from generation to generation within a single vendor, and even more when considering the disparate architectures from different vendors.  They have all the complexity and more of CPUs, but with none of the convenient compatibilities.  If you are programming for an x86 CPU, you target the same set of instructions regardless of the supplying vendor or underlying microarchitecture.  With GPUs, it is precisely the microarchitecture that gets exposed.  In addition, as part of the system software rather than application software, GPU drivers – as well as those for other devices – are expected to be “trusted”.  They should follow the principle of “first, do no harm”.  Graphics is possibly the subsystem in which this principle has been violated the most.  With graphics, the principle is more one of “first, do not be slow”; the pixel might be “blue enough” if its value can be computed in three-quarters of the time of the correct value.  The issue is where the speed-ups come from.  One possibility is an algorithmic shortcut, but drivers also play fast and loose with buffer mappings, granting direct access to potentially protected device memory in order to ensure that the fastest path to pixels is taken.  There is potential here for real evil, particularly if that device memory is adjacent to, say, a piece of your file system.  The advent of hardware memory management units has alleviated much of this sort of concern, but it is certainly possible for a poorly written driver stack to trash a perfectly good system.

So, given a community with the right sort of experience and low-level knowledge, it is certainly conceivable that you could end up with a serviceable driver stack.  For one “family” of one GPU (GPUs tend to go through iterative spins of the same basic architecture between radical revamps of the whole thing).  What about all the other ones?  A clever community might even design a driver stack architecture that allows them to change only certain pieces to support either a new underlying GPU or a different overlying API.  In fact, as we know, this is precisely what has happened (i.e., Gallium 3D).  It seems like we’re back to the automatic “great idea”, right?  Maybe.  The problem is that there are two sides of this story and the two sides don’t typically see eye-to-eye.  The GPU vendors have experience and low-level knowledge but are reluctant just to let their cleverness go off into the universe to live a life of its own.  The community has the best intentions and want to grant access to all of this great stuff, but they don’t always have the right knowledge and experience (of course, this situation improves progressively over time).  Not long ago, I was asked about this situation by a senior technical manager at a GPU vendor.  When I thought about the questions posed, I realized that what is happening on both sides is part misunderstanding and part disparity of priorities.  I’ll include here the questions and my original answers to them to illustrate:

1.       Why does the Linux community want an open-source GPU driver?

There are really two pragmatic answers to this question: 1) security 2) debugging.  Given that a driver stack is made up of a kernel component, which, technically must be licensed under the GPLv2 on Linux, and a user-space component (e.g., the EGL and OpenGL ES libraries that an application links with), these two issues have their weakest point at this boundary if the user-space portion is closed.  It is possible, though, in the modern era, getting somewhat less likely all the time, for a GPU driver to play games with memory mapping that would allow the user-space library to overwrite, say, a filesystem, based upon a bad memory access (e.g., some off-by-one error causing a write past the end of a buffer boundary and hitting another device’s memory).  There are other scenarios around this, but this is the gist.  Then there is the issue of what to do when a bug is encountered.  If the same closed-source library does something bad that crashes the system, or simply puts the wrong results on the screen, the maintainer and other developers of an open-source component like DRM (the kernel subsystem that is responsible for GPU resource management) have no recourse to figure out what went wrong, let alone address the bug.  Now they must rely on the mercy of the closed-source developers to address the problem.  Of course, there’s really also a third non-technical reason: the basic ideology of free open-source software itself.

2.       What exactly does that [producing an open-source driver] involve?

Again, it’s a two-fold answer and it’s relatively simple in concept, though a lot of work in practice, especially if you’ve developed your drivers in a proprietary environment.  One writes (or ports) a driver stack according to the architecture and coding standards of the relevant components.  In the case of the graphics stack, these are: DRM for the kernel bits, X.org for the X11 driver, and Mesa – preferably the Gallium3D model – for the EGL and OpenGL ES libraries.  Then you submit these pieces as patchsets to the relevant developer lists (each project has one), wait for the developers on the lists to pelt you with virtual rocks and garbage, revise your code based upon the comments, then resubmit as necessary until the maintainers will accept your patchsets.  Once that happens, it is part of the open-source “upstream” and anyone can work on it.  The other way is to release the hardware specifications and foster a community project to write drivers based upon the documentation.  This is actually the ideal as it empowers the community to code the drivers based upon the same documentation that vendor engineers would use; it has the added advantage of being cheaper and less work for vendor engineering teams.  Of course, from the vendor perspective, there are a lot of stakeholders to convince (including the IP lawyers) that doing so will not bring harm to the company (e.g., law suits by patent trolls).

3.       Why doesn’t graphics engineering want to do it?

Porting a proprietary driver costs a lot of time and resource, and it ties the vendor engineering team to a development effort over which they potentially have no control.  As I said, once your driver stack is open-sourced and part of an upstream repository, anyone can work on it, and the driver may be taken in a direction the vendor team does not like; worse yet from their perspective, their own patches may be refused.  There is a potential “gotcha” here as well.  For a product driven entity (the hardware vendor), the sense of priority for open-source driver development may not seem a great fit.  Of course, if someone from the vendor is positioned as the driver maintainer (Intel seems to benefit from this approach), there is a better chance of the stack going in a desired direction.  The reality is also that graphics IP vendors are concerned with supporting operating environments apart from Linux.  So, if they can’t use the open-source driver to support, say, Windows, it looks a lot like double the effort to support multiple platforms.  This, as well as the original engineering investment in the driver stack, is largely why NVIDIA, AMD, and IMG, for example, still do proprietary drivers.

4.       Who else in GPU land (e.g., IMG, NVIDIA, etc.) does this?

These examples don’t manage their driver stacks this way, but others do to varying degrees.  AMD release specs and foster the open-source driver efforts.  Of course, they also release a proprietary driver at the same time, which is their “official” support.  Intel is really the king of this approach, as they release specifications and actually use the open-source drivers as the officially supported Linux software for their hardware.  Many maintainers of graphics-related open-source components are Intel employees, and they are _very_ active in the community.  Their model would be good if you choose to go that route as they have an Intel driver maintainer that feeds into each of the core upstream components (this helps them govern the direction that their driver development takes – see my comments above under question 3).  Of course, the glaring exception here is that neither of them do this for their latest generation.  Also of interest is the Nouveau project which reverse engineers NVIDIA GPUs with absolutely no public contribution from them (just an abundance of available hardware).  This is something you could expect to see when enough low-cost developer boards with a given GPU become available.

Since this e-mail exchange, I’ve had a little additional time to ponder some of these questions a little further.  Much of what I wrote was considered from the perspective of the common Linux graphics stack; i.e., the answers are likely to be somewhat different for Android.  For question 1, there is another answer, or, rather, there is a wider answer than “debugability”.  In particular, I think the overarching theme is maintainability.  This covers not just the two parts of my original answer, but also the ability of a community to fix, improve, and extend the life of device support well beyond the cycle intended by the vendor.  I also realize that the issue of patents and their impact on the open driver situation, not to mention open software in general, has had little more than a glossing over here.  I wish I could do the topic justice.  Some of my venerable colleagues have posted excellent positions on the topic.  In particular, Kiko’s LCA2011 presentation, and Dave Airlie’s well written blog post take valiant stabs at addressing this.  My personal feeling is that in the discrete PC space, the IP vendors live in a state of “mutually assured distruction” that keeps them from poking too deeply at GPU patent litigation.  The embedded/mobile space has not yet reached this stasis, and so the fear of litigation outweighs the potential benefits of opening things up as has been done for discrete GPUs.

So what is the lesson here?  On the community side, we may have the greater challenge.  We can argue until we’re blue in the face about all of the great benefits of community development, but we can’t indemnify graphics IP vendors against lawsuits when the details of their designs are revealed in open-source drivers.  This obviously isn’t guaranteed to happen, but it only has to happen once to foul up the works.  On the vendor side, with a little help, trust and understanding you get a Linux driver stack, lots and lots of testing on a variety of platforms and an active community supporting your platforms.  This isn’t a free lunch, though, and it is important to seed the development community with engineering resources; this improves the community as well as the software it produces.

Recent Posts

Showing 13 comments

  • Matt

    You’ve forgotten one of the main reason users (people who are not kernel developers) want an open-source driver: Portability.

    One example of this is the Intel Poulsbo chipset. It had fairly decent performance, but it could only be used with a closed binary blob on certain Ubuntu distributions. Vendors never maintain support for a device for very long, as they’re always trying to sell new products. The Fit-PC had a Poulsbo chipset and great performance/watt, but you’d be stuck on one flavour of Ubuntu for the rest of your life. Unacceptable.

    Or on embedded devices (I work a lot with OpenWRT) we like to install different programs and really utilize the hardware. It’s only recently that manufacturers started migrating away from the 2.4 kernel in routers and modems. Broadcom’s been terrible about drivers for its wireless chips until recently. Porting stuff back to 2.4 is a pain, and we don’t want to run stuff that old.

    Technical people want control over their hardware. It’s not all about security. Sometimes we just want to make it do new things, and since the kernel ABI changes with each release, we can’t unless you write your driver with an eye to the future. Computing evolves, and closed drivers keep us in the stone age.

    A split open/closed model is better than nothing, but at least let me compile it with whatever kernel I want and put my own distro on there. I have hundreds of dollars of hardware that’s useless because the manufacturers didn’t support it and didn’t share any information. I now refuse to buy anything more expensive than a meal or two if I can’t use the hardware myself in the way I want to.

    Also, the kernel developers do a great job of doing most of the work themselves once you get it in there. IP trolls are a real concern, but saying you want to “control the direction of the driver” is silly. Consumers want it to work, and that should be your goal. If they’re hurting performance, let them know, but if they make your hardware do something you’ve never thought of, be proud. Your technology is to some degree future-proof.

    Technical users upgrade hardware all the time for performance. That’s considered a good reason for an upgrade, and you don’t regret the money. Poor vendor support should never be a reason to upgrade. It should be a reason to change vendors.

  • Mark Ransom

    You missed one factor as an impediment to open source drivers – the legal obligations required for Windows and HDCP compatibility. The video chip manufacturers are prohibited from divulging the documentation and code that would enable bypassing of copy protection. I first learned of this back in 2006: http://discuss.joelonsoftware.com/default.asp?joel.3.431446.17

    • Matt

      Somehow I think that’s going to be far less important with the master HDCP key in the wild and HDCP having been “cracked”.

      It was never terribly secure, since you could pull an HDCP chip out of a used TV and build a device for stripping the DRM from the signal. Now all you need is a ~$250 FPGA.

      Heck, even half of the TVs run Linux. I sent in a request for the source code of a broken HDTV. After I got it, I opened it up to find out that it’s got an FPGA and an ARM cpu inside.

      While the public doesn’t like it, we’re used to firmware and can deal with it. If there’s a trade secret that absolutely has to be protected, it can be split off into a closed binary. It’s an upfront cost, but the benefits are there.

  • Bill

    Maintenance !

    Closed source drivers are a real pain when it comes to the embedded space. I have been working with an SoC vendor on a bug that I found and even they do not have access to the source code to their closed source video firmware. After many delays, we eventually found out that the SoC Vendor’s maintenance contract with the firmware supplier had expired before we could bring our product to market.

  • Adam Williamson

    I’m somewhat confused by the reference to “Of course, the glaring exception here is that neither of them [Intel and AMD] do this [develop / encourage open source drivers] for their latest generation.”

    The only Intel graphics hardware which doesn’t have an open source driver is the infamous Poulsbo series, including the GMA 500 and the newish GMA 600. I guess you might technically refer to the GMA 600 as Intel’s ‘latest generation’, but it seems a bit misleading: the Poulsbo line is kind of a special case in and of itself and you can’t really consider it together with the rest of Intel’s graphics hardware. As far as I know, they intend to continue providing open source support for their non-Poulsbo adapters – as seen in typical laptops – just as they always have.

    • Jesse Barker

      Fundamentally, my point was that even those vendors who play nicely with open-source do not do so universally. You do hit upon an interesting point which is that SoC vendors do not in many cases produce their own GPU; they buy it from another IP vendor. The support for GMA{5,6}00 is of particular interest to Linaro members as Imagination GPUs are quite popular in the ARM ecosystem and anything that can be done to urge better open-source support for them is a win for everyone.

  • Ken

    Good points.

    For little ol’ me (user, not graphics/kernel developer), the big reason is reliability, which is likely related to debugging/maintenance. Of the half dozen+ graphics cards I’ve used on Linux, I’ve never had a free driver fail, and I’ve never had a proprietary driver that always worked. I wouldn’t buy a graphics card today that didn’t have a free driver, even if it was cheaper upfront, because I have very little confidence that it would work well.

    I’ve heard the “mutually assured destruction” point before, but I’m still not sure I believe it. You say “we can’t indemnify graphics IP vendors against lawsuits when the details of their designs are revealed in open-source drivers”. Are the details of their designs hidden today, simply because they’re in compiled code? I would think that binary code wouldn’t be much harder to compare against than source code, if at all.

    • Jesse Barker

      For modern architectures, most of the important details are hidden in the shader compiler, so it’s not the binary drivers, per se, that are obfuscating the architectural details (e.g., seeing the implementation of “glDrawArrays” wouldn’t tell you very much).

      • Justin Barwick

        Jesse,

        I have a team trying to develop open source drivers for the Ingenic jz4770 processor
        specifically for the gpu which is Vivante GC860 to not much avail. We could use your assistance
        in this matter if you had free time for advice.

        Regards,

        Justin

        • Jesse Barker

          Hi Justin,

          I would be happy to chat with you about your efforts. The Linaro graphics working group page is here:

          https://wiki.linaro.org/WorkingGroups/Middleware/Graphics

          And you can click on the “People” link in the banner at the top for contact info.

          cheers,
          Jesse

        • Wladimir

          I’m currently working on reverse engineering the Vivante GCxxx drivers, and am already pretty far along in figuring out the state bits, and shader ISA, and how everything fits together. I expect to be able to do basic rendering without the binary blob soon. I’d like to eventually make an open source driver or at least document my effort well enough so somebody else can.
          Please contact me on witchspace81@gmail.com if you’re interested in sharing information.

        • Wladimir

          Work has been started on an open source driver for the Vivante GCxxx GPUs, see https://github.com/laanwj/etna_viv . It is still in the experimental stages but it can render smoothed RGB cubes at any resolution already :-)

  • Guillaume FORTAINE

    Hello,

    For your information :

    http://limadriver.org/

    “Lima: An open source graphics driver for ARM Mali GPUs”

    Best Regards,

    Guillaume FORTAINE