create blog

go home go home
  1. about
  2. code
  3. wiki
  4. blog

Archive for the ‘Testing’ Category

JavaScript Animation Benchmarks

Wednesday, October 21st, 2009

I’ve been working on a mixin to add animation to SproutCore views.

The current version only works for layout properties, and does not yet work for centerX and centerY properties (they used to work, but some of the performance optimizations have made it slightly more tricky—I’ll be adding it back soon, though).

I decided to see how fast the code was in different browsers. The tests were done using a test application which generated a specified number of views, and then, once per second, updated a “frames per second” display. The measuring is somewhat subjective, as I have to deduce, based on consistency (or lack thereof) in the numbers, what the frame rate actually is. For the most part, they were pretty consistent, but the WebKit browsers at really low numbers of views (and really high frame rates) could be quite inconsistent at times.

I ran the tests on two separate machines:

  • Mac OS X Snow Leopard on MacBook Pro 15″ Core 2 Duo 2.33Gz w/2GB RAM. Lots of open programs, including iTunes. Not as scientific as would be optimal, but it is my work machine.
  • Windows XP on Pentium 4 3.20GHz w/2GB RAM. Only the browser was open.

Browsers tested (note that my Firefox was OLD):

  • Safari 4.0.3 for Mac
  • Firefox 3.5.3 for Mac
  • Chromium 4.0.223.3 (29380) for Mac
  • Internet Explorer 8
  • Firefox 3.0.1 for Windows (oops. Should have upgraded first. Remind me to update numbers on Friday)
  • Safari 4.0.3 for Windows
  • Chrome 3.0.195.27 for Windows

Here is the data:

Data collected during testing of the different browsers

Data collected during testing of the different browsers

That’s boring. Where are the pictures?

All Browsers

All Browsers

Even here, you can tell that Chrome and Chromium are the best performers for small numbers of views. However, for larger numbers of views, Safari on Mac was a very clear winner, running at 12 frames per second for 2000 LabelViews..

Predictably, Internet Explorer was slowest. I didn’t bother to run all the tests for it.

Windows Browsers

Windows Browsers

This chart shows only the Windows browsers. Again, you can tell IE is very slow. Chrome is very very fast. Safari is pretty good. My out-of-date Firefox is not always that much faster than IE; it will be interesting to see how Firefox 3.5 fares.

Notice the dip in performance for Safari at around 300? It briefly does worse than Firefox. It quickly levels off, though, and is second only to Chrome.

Mac Browsers

Mac Browsers

And, of course, the best for last. Note how we see the same dip in performance for Safari—though it does not dip below Firefox; it would have dipped below Chromium, were it not already below it. Again, though, it leveled out faster than Chromium.

I always wanted to do a performance chart, but now that I have… it’s a lot of work (about 57 or so data points there… each measured separately).

Question Is the difference between Safari and Chromium completely caused by Google’s V8 JavaScript engine, or are there a few other differences?

PDF Parsing

Tuesday, May 19th, 2009

Just a little status update: yesterday, I finally managed to make the new PDF engine pass all PDF-related tests in the Create Framework. It is revision 100 in the pdf-parsing branch on Launchpad.

Yes, all of the tests have been updated, but they were hand-checked first. As the new engine handles the timing of the writing of objects differently, the tests will work differently. They will change further, in fact, because I just realized that the maximum size for a page tree node is still set to 3 (for debugging purposes) — which is not at all what the final size should be — so the tests that have more than three pages are, unfortunately, wrong. Further, it is possible that some areas of the Create Framework may even have some leaks regarding PDF, so fixes to those could cause problems. This is not a stable branch, is not meant to be a stable branch, and is not guaranteed to even be in a compilable state. I’ve disclaimed, so if your computer blows up, it is not my fault.

Now, I’ve finally started PDF parsing. Currently, I’ve got the first XRef table being read in. I’m excited. Hopefully, by the end of the week, I’ll have some form of PDF-in-PDF embedding working. Currently, I’m aiming to support only PDF 1.4 — before all of that cross-reference stream and object stream business came about that would require the ability to handle compression (which, while on the eventual to-do list, is not currently a priority).

Regression Tests

Tuesday, February 17th, 2009

The framework finally became complete enough to where I felt like I could add tests. Perhaps I should have started with tests, but the tests themselves require a composition of many of the components, so it would have been impractical.

Implementing tests had a very nice side-effect. Tests are meant to test expected behavior. As it turns out, there were a lot of areas where the framework did not behave as expected. I fixed at least seven bugs over the course of the two days when I wrote the current four tests. I say “at least” because I often fix bugs as I see them and promptly forget about them, or group them into larger categories when I write the revision release notes.

In short, just by writing a few tests, I drastically increased the system’s stability. And, since they will run often, they will ensure continued stability as well. Short term and long term, testing can be great.