Connected TV development with AIR for TV

Posted on by
Reply

Having just finished building the UI for the YouView set top box, I thought I’d share some of my insights into best practices when building applications for such resource constrained devices. The YouView UI is AIR based, written in AS3 and runs in Stagecraft 2, also known as ‘AIR for TV’. As the name suggests, AIR for TV is a special version of the Flash player for embedded systems, such as set top boxes. The first incarnation of the YouView UI (back when it was just codenamed ‘canvas’) was for Stagecraft version 1, which means coding in AS2 and suffering the abysmal performance that comes with running on AVM1 (ActionScript Virtual Machine 1).

Despite the delays and the need to code the UI from scratch in AS3, I think it was ultimately the right decision. Stagecraft 2 is a much better platform – Stagecraft 2.5.1 to be precise. It was a great opportunity to learn how to write optimal code and use hardware acceleration effectively on resource constrained devices. I’ll be doing some tutorials on this in the near future, but here’s the key points to observe when developing for such platforms:

  • Limit the complexity of your display list heirarchy
    This may sound obvious, but ensure you nest as few things as possible, keeping the display list as shallow as possible. Stagecraft needs to traverse through the display list, working out which areas of the screen to redraw. This is similar to how the desktop Flash Player handles redraws, but with some key differences to how it decides what needs redrawing, how it tackles moving display objects and how it delegates the work of updating the frame buffer – a subject for another time. Mostly importantly, if you’re developing for a resource constrained device (such as mobile or set top box), you’ll have very limited CPU power, even if the device’s GPU (graphics processing unit) affords you great hardware acceleration capabilities. So, before Stagecraft can delegate any work to hardware, it enumerates changes in the display list in software. Complex display list heirarchies are a headache for some of the low-powered CPUs found in mobiles and set top boxes and this’ll show up as rocketing CPU usage, low framerates and few spare ‘DoPlays’ in Stagecraft (spare work cycles). By keeping your display list shallow, with only the bare minimum of display objects on stage at any one time, you’ll be making life easier for Stagecraft by doing less work on the CPU – whether or not graphics are drawn in software or hardware.

  • Benchmark everything
    When building an application for a resource constrained device, you should be able to run each component in isolation, to assess its drain on CPU and system/video memory. There’s no point optimising the hell out of one component, when it’s actually another one that is the source of your performance bottleneck.

  • Know thine hardware acceleration capabilities
    There’s no point blindly using cacheAsBitmap and cacheAsBitmapMatrix everywhere, if it’s not going to speed things up on the target device. Worse still, too many cacheAsBitmaps and you may be just wasting valuable video memory, or causing unnecessary redraws (again, the subject of a future article). A lot of platforms will accelerate bitmaps, even if stretched, but not necessarily if flipped or rotated. Alpha on bitmaps (or anything cached as bitmap) will usually be accelerated too, but this is not necessarily the case with all colour transforms. Benchmarking any component you’re building will quickly tell you where you might have pushed it too far, but you should also have a way of verifying that a particular set of transforms is indeed hardware accelerated. Stagecraft provides this when using its –showblit command line parameter. I’ll be going into more detail about this in another post.

  • Mind your memory
    When using various hardware acceleration tricks, especially on resource constrained devices, video memory is at a premium and usually in limited supply. You will need to know the limits and have a way of seeing how much video memory your application is using at any one time – ensuring you dispose and dereference any bitmaps you’re finished with too. If your platform uses DirectFB for its rendering, as YouView does, the executable ‘dfdump’ can show you just where your video memory is going. This is something else I’ll get into in another article.

  • Blit blit blit
    This refers to blitting, where blocks of pixels are copied from one bitmap to another. This technique is used a lot in games, where graphics performance is critical, you should arm yourself with the basics of how old video games used blitting of multiple things to a single bitmap for performance and video memory efficiency.

I’ll probably go into more depth on each of these things in forthcoming posts. Stay tuned.

Chinese Handwriting Recognition App

Posted on by
2

Chinese Handwriting Recognition App

 

 

 

 

 

 

 

Unlike the iPad, the BlackBerry PlayBook has rather poor international keyboard support, with no method for entering chinese characters. I like the way iOS achieves this, so went about building my own version in ActionScript.

This was mainly an academic exercise and to help me to learn to write chinese. My approach was to sample the strokes drawn into the app as a series of up to 8 directions, including the relative position of a given stroke to the previous stroke, again as one of 8 directions. This pattern, represented as a string of numbers is then put through a smoothing algorithm, to remove some unnecessary noise and then compared with a dictionary of pattern keys, which may contain one or more suggested characters. If there are no hits, an advanced search occurs, by mutating the given pattern in specific ways, in order to find alternative suggestions. I can also find characters based on the next most likely character to the one you’ve just entered, using frequency analysis on given sample text.

The app will eventually be a PlayBook App, but is still unfinished and currently in ‘training mode’, so that character patterns can be trained into the database. It’s currently primed with some simplified sample data, from which it picks the most popular few characters to learn. If you write chinese, give it a go.

Symform – a truly distributed cloud storage service

Posted on by
Reply

I recently discovered Symform, a unique type of cloud storage service, which gives you up to 100GB of free cloud storage. What’s special about them is that your data isn’t stored in one Data Centre somewhere, but encrypted and distributed amongst thousands of other Symform users’ harddrives, offering better redundancy than RAID storage. You can contribute space to cloud to receive more storage and there’s the potential to upgrade to unlimited storage.

SmoothBitmap – How to enforce pixel smoothing on a Bitmap

Posted on by
Reply

A common oversight when using Bitmaps with loaded content is that Flash will revert a Bitmap’s smoothing parameter to false when you replace its bitmapData. It’s simple enough to fix, but since you may not know if someone is going to replace the bitmapData of a Bitmap you have created – then it’s often better to code defensively around it.

This little SmoothBitmap class is for just such an occassion. Instantiate it like a regular Bitmap and, no matter what another developer does with it, smooth pixels when scaling/rotating will be ensured.

CODING WRONGS – Where do I start with the bad?

Posted on by
Reply

It gets scary out there sometimes. During my freelance career I've worked a lot of different companies and have seen such coding horrors as you cannot imagine. So I thought I'd start immortalising some of them - so that we can all learn better coding practices, by looking at the bad.

Starter for 10 - What's wrong with this picture?

public function set someProperty(value:String):void
{
    var newPixels:BitmapData = new BitmapData(someUint, someUint, true, 0);
    someBlitUtil.blitStuffTo(newPixels, value);
    someBitmap.bitmapData = newPixels;
}

If it's not immediately obvious, the fubar here is the potential replacement of a Bitmap's BitmapData, without explictly disposing any existing BitmapData. I see this kind of thing quite often and it's the source of many a memory leak. AVM2 isn't that great and dealing with this kind of situation and there's a crucial difference between GC cleaning up out-of-scope objects for you and things like BitmapData: GC will reclaim the memory associated with objects 'when it feels like it', whereas explicitly calling the 'dispose' method of a BitmapData will immediately give you back that memory. In the case of hardware accelerated setups, the memory associated with the pixel data itself (video memory) will be reclaimed immediately. You shouldn't make more work for GC when you can avoid it - defensive programming is always best!

TextField.getRawText() what it does

Posted on by
Reply

I was recently creating an API that required extending TextField and happened across the getRawText() method. I assumed this returned the text from the field without formatting or something - so I looked up the AS3 docs for flash.text.TextField.

Nothing there - gee thanks Adobe. A quick search turned up this which, it turns out, isn't quite accurate.

So, with a tad of testing, it appears that getRawText() returns the text, stripped of any HTML tags (if you had set htmlText). I now wonder if this is faster than using a RegEx to strip the tags and why Adobe didn't document it?