Programmers Stack Exchange is a question and answer site for professional programmers interested in conceptual questions about software development. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I'm working on an app that involves real-time manipulation of vector paths at 60fps, and I'm very surprised by how little information there is on the subject. At first, I tried to implement my idea using CoreGraphics, but it didn't perform adequately for my purposes. I then discovered that there was a Khronos standard for hardware-accelerated vector graphics called OpenVG, and thankfully a kind soul had written an OpenGL ES semi-implementation called MonkVG.

But despite the fact that OpenVG is a very practically useful API, it seems more or less abandoned by Khronos. According to Wikipedia, since 2011, the working group "decided to... not make any regular meeting [sic] for further standardization". The documentation, best I can find, consists of just a single reference card. And what's more, there are barely any examples of OpenVG anywhere on the internet. I can find hundreds of OpenGL tutorials in the blink of an eye, but OpenVG seems conspicuously missing.

You'd think that hardware-accelerated vectors would be more important in today's world of rapidly-increasing resolutions, and it does seem that many companies are implementing their own ways of doing this. For example, Qt and Flash have schemes for hardware-accelerated vectors, and many of Adobe's tools have hardware acceleration as an option. But it seems like the wheel is getting reinvented when a standard already exists!

Is there something I'm missing about OpenVG that makes it unsuitable for real-world use? Or is it just that the standard didn't catch on in time and now it's destined for obscurity? Do you think there's room for a standardized API for hardware-accelerated vector graphics in the future, or will it just be easier to use traditional raster-based techniques? Or are vectors simply on their way out, before they were ever in?

share|improve this question
Before you downvote this question, please remember that subjective questions are allowed on Programmers, so long as they are constructive, which I think this one is. – Archagon Mar 21 '13 at 18:14
I upvoted because it doesn't seem like a bad question.. – The Muffin Man Mar 21 '13 at 18:18
It's interesting to note that computer graphics started out as Vector Graphics. Including displays. – Clockwork-Muse Mar 21 '13 at 18:52
Off the top of my head, I'd recon OpenVG failed because the industry didn't trust it after the debacle that happened with OpenGL. I'm too lazy to do research to back up that theory, so I'll leave it as a comment. – Michael Brown Mar 21 '13 at 19:41
@Earlz - directly from the FAQ: -- see second section – DXM Mar 21 '13 at 20:00

update: See bottom of reply

This answer comes a bit too late, but I hope to shine light to others (particularly now that C++ standard committee wants to incorporate Cairo into std):

The reason nobody really cares about "accelerated vector graphics" is because of how GPUs work. GPUs work using massive parallelization and SIMD capabilities to colour each pixel. AMD typically works in blocks of 64x648x8 pixels while NVIDIA cards typically work in 32x32 4x4 pixels [See update at the bottom]

Even if they're rendering a 3D triangle, the GPU works on whole quads that this triangle covers. So if a triangle doesn't cover all 8x8 pixels in the block (or 4x4 in the case of nvidia) the GPU will compute the colour of uncovered pixels and then discard the result. In other words, the processing power for uncovered pixels is wasted. While this seems wasteful, it works incredibly good for rendering large 3D triangles when paired with a massive number of GPU cores (more detailed info here: Optimizing the basic rasterizer).

So, when we look back at vector based rasterization, you'll notice that when drawing lines, even if they're thick, there is a massive blank space. A lot of processing power wasted, and more importantly bandwidth (which is the major cause of power consumption, and often a bottleneck) So, unless you're drawing an horizontal or vertical line with a thickness multiple of 8, and it perfectly aligns to 8 pixel boundaries, a lot of processing power and bandwidth will be wasted.

The amount of "waste" can be reduced by calculating the hull to render (like NV_path_rendering does), but the GPU is still constrained to 8x8/4x4 blocks (also probably the NVIDIA's GPU benchmarks scale better with higher resolutions, the pixels_covered / pixels_wasted ratio is much lower).

This is why many people don't even care about "vector hw acceleration". GPUs simply aren't well suited for the task.

NV_path_rendering is more the exception than the norm, and they've introduced the novel trick of using the stencil buffer; which supports compression and can significantly reduce bandwidth usage.

Nonetheless, I remain skeptic of NV_path_rendering, and with a bit of googling shows that Qt when using OpenGL (aka the recomended way) is significantly faster than NVIDIA's NV_path_rendering: NV Path rendering In other words, NVIDIA's slides were "accidentally" comparing XRender's version of Qt. Ooops.

Instead of arguing that "everything vector drawing with hw acceleration is faster", Qt developers are more honest admitting HW accelerated vector drawing is not always better (see how their rendering works explained: Qt Graphics and Performance – OpenGL)

And we've not touched the part of "live editing" vector graphics, which requires triangle strip generation on the fly. When editing complex svgs, this could actually add serious overhead.

Whether it is better or not, it highly depends on the applications; as to your original question "why it hasn't taken off", I hope it is now answered: there are many disadvantages and constraints to take into account, often making a lot of people skeptical and may be even biasing them into not implementing one.

update: I've been pointed out the numbers are completely off base, as the mentioned GPUs don't rasterize in 64x64 & 32x32 blocks but rather 8x8 = 64 and 4x4 = 32. This pretty much nullifies the conclusions of the post. I will soon update this post later with more up to date information.

share|improve this answer
Kilgard has responded to that blog post of Zack Rusin at the very end of the comments. There was driver bug in the version that Rusin tested. The newer 3xx driver beat Rusin's code by a factor of 2x-4x . Rusin has not responded after that. – Fizz Aug 6 '14 at 1:00
Also note that there's now work being done in Skia (Google Chrome/Chromium's 2D library) to support NV_path_rendering: The issue is kinda complicated because OpenGL ES is not entirely compatible with NV_path_rendering. – Fizz Aug 6 '14 at 1:18
According to the much newer presentation at… there's also work on adding NV_path_rendering to Illustrator. It also says that Skia already uses NV_path_rendering when available (although the bug report I linked in my previous comment suggests this doesn't work as well as one might hope.) – Fizz Aug 6 '14 at 1:57
Also Chris Wilson (a cairo developer and Intel employee) had only good things to say about NV_path_rendering; it's basically on cairo's wishlist: – Fizz Aug 6 '14 at 2:31

I don't think it is really true that nobody really cares about "accelerated vector graphics" as written in this answer.

Nvidia seems to care a fair bit. Besides Kilgard who is the lead technical guy on NV_path_rendering (henceforth NVpr to save my fingers), the Khronos president, Neil Trevett, who is also a VP at Nvidia, has promoted NVpr as much as he could in past couple of years; see his talk1, talk2 or talk3. And that seems to have paid off a bit. As the time of this writing, NVpr is now used in Google's Skia (which in turn is used in Google Chrome) and independently [of Skia] in a beta version of Adobe Illustrator CC (beta), according to Kilgard's slides at GTC14; there are also some videos of the talks given there: Kilgard's and Adobe's. A Cairo dev (who works for Intel) also seems interested in NVpr. Mozilla/Firefox devs also experimented with NVpr and they do in fact care about GPU accelerated vector graphics in general as this FOSDEM14 talk shows.

Microsoft also cares a fair bit because they created Direct2D, which is used fairly widely [if you believe the Mozilla dev from the aforementioned talk].

Now to get to the point of the original question: there are indeed some technical reasons why using GPUs for path rendering is not straightforward. If you want to read about how path rendering differs from bog-standard 3D vertex geometry and what makes GPU acceleration of path rendering non-trivial, then Kilgard has a very good FAQ-like post, which is unfortunately buried somewhere in the OpenGL forum.

For more details on how Direct2D, NVpr and such work, you could read Kilgard's Siggraph 2012 paper, which of course is focused on NVpr, but also does a good job surveying prior approaches. Suffice to say that quick hacks don't work too well... (as the text of the PSE question noted.) There are significant performance differences between these approaches as discussed in that paper and shown in some of Kilgard's early demos, e.g. in this video. I should also note that official NVpr extension document details several performance tunings over the years.

Just because NVpr wasn't so great on Linux in 2011 (in its first released implementation), as that 2011 blog post of Qt's Zack Rusin said, it doesn't mean that GPU acceleration of vectors/paths is hopeless as Mr. Goldberg's answer appears to have inferred from that. Kilgard has in fact replied to the end of that blog post with updated drivers showing 2x-4x improvement over Qt's faster code and Rusin hasn't said anything after that.

Valve Corp. also cares about GPU-accelerated vector rendering, but in a more limited way, relating to font/glyph rendering. They've had a nice, fast implementation of large font smoothing using GPU-accelerated signed distance fields (SDF) presented at Siggraph 2007, which is used in their games like TF; there's a video demonstration of the technique posted on YouTube (but I'm not sure who made that).

The SDF approach has seen some refinements by one of the Cairo & pango devs in the form of GLyphy; its author gave a talk at 2014. The too-long-didn't-watch version is that he does an arc-spline approximation of the Bezier curves in order to make the SDF computation more tractable in vector (rather than in raster) space (Valve did the latter). But even with the arc-spline approximation, the computation was still slow; he said his first version ran at 3 fps. So he now does some grid-based culling for stuff that's "too far away", which looks like form of LOD (level of detail) but in the SDF space. With this optimization his demos ran at 60 fps (and it was probably Vsync limited). However his shaders are incredibly complex and push the limits of hardware and drivers. He said something along the lines of: "for every driver/OS combination we had to change things". He also found significant bugs in shader compilers, some of which were then fixed by their respective devs. So it sounds a lot like AAA gaming titles development...

On another tack, it appears that Microsoft has commissioned/specified a little bit of new GPU hardware to improve their Direct2D implementation with, hardware which is used by Windows 8, if available. This is called target-independent rasterization (TIR), a name which is a bit misleading as to what the stuff actually seems to do, which is spelled out in Microsoft's patent application. AMD claimed that TIR improved performance in 2D vector graphics by some 500%. And there was a bit of "war of words" between them and Nvidia because Kepler GPU's don't have it, whereas AMD's GCN-based GPUs do. Nvidia has confirmed that this is indeed a little bit of new hardware, not simply something a driver update can provide. Sinofsky's blog post has a few more details, including some actual benchmarks of TIR. I'm quoting only the general idea bits:

to improve performance when rendering irregular geometry (e.g. geographical borders on a map), we use a new graphics hardware feature called Target Independent Rasterization, or TIR.

TIR enables Direct2D to spend fewer CPU cycles on tessellation, so it can give drawing instructions to the GPU more quickly and efficiently, without sacrificing visual quality. TIR is available in new GPU hardware designed for Windows 8 that supports DirectX 11.1.

Below is a chart showing the performance improvement for rendering anti-aliased geometry from a variety of SVG files on a DirectX 11.1 GPU supporting TIR: [chart snipped]

We worked closely with our graphics hardware partners [read AMD] to design TIR. Dramatic improvements were made possible because of that partnership. DirectX 11.1 hardware is already on the market today and we’re working with our partners to make sure more TIR-capable products will be broadly available.

I guess this was one of the nice things that Win 8 added that was mostly lost to the world in the Metro UI fiasco...

share|improve this answer

There are some fine answers here but I wanted to echo the same basic idea, yet with a twist.

GPU rasterization is actually pretty amazing at rendering epic amounts of "homogeneous" primitives. For example, I don't even have a very good GPU (GTX 560 Ti) and can rasterize 4 million wireframe lines in 3D through a vertex and frag shader at over 30 frames per second through the GPU.

It would be difficult (if even possible) for a CPU-based rasterizer to come even close despite the amount of conceptual work wasted by the GPU. The GPU just chews through these primitives like nothing, even though it's not even designed to really do this (ex: shader and matrix transforms are wasted with just 2D rasterization).

But the keyword is "homogeneous". Once you start mixing a variety of primitives that all require different shaders to rasterize, possibly even some textures here and there, it's suddenly exponentially harder to get decent performance. Texture images might need to be coalesced into texture atlases, like sprite sheets, with the primitives also sorted to minimize context switches for shaders and textures. Just getting nice subpixel antialiasing on rendered lines required a cutting-edge research paper previously. It might be possible to get something amazing out of that after a whole lot of proprietary work, it's just really hard, and for some desktop application like a web browser or a word processor's GUI, that's a whole lot of work for something that was never the main bottleneck in the the first place. Naive attempts like converting a bezier curve to line strips is also likely to cost more than it saves than rasterizing it directly on the fly with De Casteljau, and it definitely hogs up more memory to have to store all this state.

Vector graphics are awesome and featherweight, I wished we used them more (I can imagine all kinds of applications requiring considerably less disk space and install time, for example, if they used vector graphics). But we don't necessarily need a GPU pipeline to render them quickly.

Another thing is that people tend to underestimate the power of today's CPUs. They are freaking beasts of their own. I actually beat my colleague's GPU code rendering epic bloom over NVidia titans by implementing the bloom filter on the CPU.

enter image description here

His GPU bloom was faster than mine for a really small blur radius, like 3 or 4 pixels. Where mine outperformed it was for bigger amounts of blur, like 20+ pixels, and in that screenshot, ridiculous 400-pixel radius blurs at over 60 frames per second on the CPU.

A lot of people seem to think the CPU can't do this stuff when it can. It's not even difficult, it took me less time to write that blur filter than my peer, who is far better at GPU shaders than me, took to write his deferred frag shader. All I did was multithread the image filter and use SIMD and a tiled memory access pattern for cache efficiency.

So I don't know if that's at all a fair comparison of CPU vs. GPU, maybe my colleague's blur shader wasn't so great. But judging by the surprise people had here, I think a lot of people underestimate the power of the CPU while overestimating the power of the GPU. It's easy to read numbers like 2000+ cuda cores and think that's 500+ times better than an i7 with a measly 4 cores, but it's apples and oranges.

Another place where CPUs can actually start to compete with GPUs, counter-intuitively, is particle rasterization, where I've found it easy to render tens of millions of particles on my i7 with just CPU processing. The fill rate of all kinds of little transparent particles on the GPU doesn't seem to be that great.

I think one of the reasons GPU programming is getting so popular is that there it's more excusable to think micro -- after all, we have to for efficient GPU code. Papers published on GPU techniques focus on things like reducing the number of arithmetical ops performed per fragment -- like uber micro assembly-level thinking. CPU coding is often accompanied by a strong disapproval against micro-optimization (probably largely in part by the vast number of libraries available which have already been micro-optimized for us combined with the competence of optimizing compilers at tasks like register allocation) which has more people reluctant to try things like handwritten SIMD intrinsics, when they can be just as rewarding (if done well) from a performance standpoint. So GPU is, to me, somewhat of an excuse for people even using much higher-level languages to get all micro-level again with their efficiency focus, where they might experience some of the same benefits they would have if they applied that kind of focus towards CPU code.

That's not a slant on GPUs though, since I'm also trying to use them more and more since there are some things they do undeniably better than CPU. But they're not necessarily better at everything, and not even necessarily at the things people often think they're better at (complex image processing with non-sequential access patterns, e.g.), and complex vector rasterization with antialiasing, gradients, drop shadows, bezier paths, things like that might not actually have the GPU providing such an edge, or at least not as much of an edge as some people might think.

share|improve this answer

protected by Robert Harvey Apr 20 '15 at 17:54

Thank you for your interest in this question. Because it has attracted low-quality or spam answers that had to be removed, posting an answer now requires 10 reputation on this site (the association bonus does not count).

Would you like to answer one of these unanswered questions instead?

Not the answer you're looking for? Browse other questions tagged or ask your own question.