A California firm holds the low-cost, high-performance key to the next generation of portable multimedia devices with its extremely power-efficient programmable parallel processing chip technology.
Why is parallel processing important, and why are we dedicating a whole feature to it in the pages of Pen Computing Magazine? Because smartphones and other mobile devices are about to get a whole lot smarter. Because mobile digital TV (MDTV) is just around the corner and it could be a multibillion dollar market in just a few years. And because one company seems to hold the key to an important part of that future. That company is NeoMagic, headquartered in Santa Clara, California, right in the heart of Silicon Valley.
Who is NeoMagic?
NeoMagic Corporation designs, develops and markets semiconductors solutions. They started back in 1993. In the past, up to 2000 or so, NeoMagic specialized in multimedia accelerators used in high-end notebook computers. They then got out of that market and began concentrating on applications processors. Those "Systems-on-Chips" (SOCs) are geared towards providing next generation, very high performance multimedia on mobile phones and handheld devices. Why the move away from notebooks to next-gen mobile phones? Because the notebook market is a maturing and rather crowded market whereas for next-gen phones, the sky is the limit. According to Credit Suisse, 822 million mobile phones were shipped in 2005. They expect that number to rise to 966 million in 2006, 1,078 million in 2007, and 1,129 million in 2008. That's many times the number of notebooks sold worldwide.
Not all of that billion-plus cellphones will have next-generation multimedia wizardry built in, but a significant percentage will. Cool features in phones sells phones -- and lucrative service plans. Digital cameras are almost a standard feature in cellphones now. And we're no longer talking fuzzy VGA snapshots. Samsung recently revealed a 10-megapixel cameraphone (the SCH-B600), and it supports Korea's satellite digital multimedia broadcasting services. The future of cellphones, and not just cellphones, is in dramatically increased multimedia that surpasses much of what until recently was only seen in sci-fi movies. We're talking super-hi-res imaging, highly compressed video using the MPEG-4 and H.264 standards, 3D graphics for sophisticated gaming, and, of course, audio.
What is an application processor?
All of this needs very powerful chips, but not just any chips. What's needed are application processors. These are very specialized and highly integrated chips, or rather the aforementioned System-On-Chips that include a CPU with instruction and data caches, a memory controller, and a variety of peripheral devices. Add the appropriate memory and input/output devices and the application processor can run user applications under a variety of common mobile operating platforms such as Windows Mobile, Symbian, embedded Linux, the next-gen Palm OS (codenamed "ALP"), etc. Application processors specialize in multimedia -- audio and video. They support Java for games and a variety of downloaded applications, and they are also strong in security features for e-commerce, digital rights management and so on.
An application processor is not a run-of-the-mill general purpose processor. In a phone, for example, or some other wireless device, that task falls to a so called baseband processor that handles the wireless aspects, headphones, speakers, microphone and related functions. The application processor, on the other hand, has its own media engine, processor, memory and handles I/O, the display, expansion slots, imaging -- everything that requires maximum performance.
The trick then becomes to have application processors that support whatever operating systems, baseband processors, and wireless communications protocols vendors (and the public!) want. And what does the public want? Ever more powerful devices that can do incredible new things. And they want that in ever smaller, ever slimmer phones. And without giving up the battery life they've become accustomed to. In fact, they expect even longer battery life.
The battery life dilemma
How can that be done? Those of us who have followed the mobile field for a while know one of the industry's dirty secrets: battery life that once was measured in weeks back in the day of the original Palm Pilot is now measured in a very few hours in most of today's state-of-the-art devices. Sure, battery technology has improved, but now a smaller battery has to power a brighter screen with much higher resolution, multimedia, much more memory, a far faster processor, and perhaps even an internal hard disk. Add to that an integrated GPS receiver, Bluetooth and WiFi, and high-speed graphics, and you can see why even improved battery technology is falling behind. As a result, manufacturers use tricks like ultra-conservative (and often obnoxiously intrusive) power management schemes, non-volatile memory so that at least you don't lose your data when the battery dies, trickle charging via USB, and so on. Crutches all.
Used to be that short battery life was blamed on ever larger color screens. But that argument doesn't wash with next-gen mobile phones as their screens are not very large. Much of the power is eaten up by the processor, so that is where the savings are to be found. But how are those savings accomplished? After all, more performance means more power consumption, and what we're after is more performance but less power consumption.
Enter NeoMagic and its bag of tricks. Tricks that not only greatly improve multimedia performance, but also reduce power consumption. One such technology is APA, which stands for Associative Processing Array. That doesn't say much to the average person and perhaps not even to most engineers, so let me explain. We've all been led to believe that the performance of a processor is measured by its clock speed. A 1.6GHz processor is better than a 900MHz processor, and one speeding along at 3.2GHz is definitely better than last year's 2.4GHz model. Sure, over the years Intel has come up with plenty of additional buzzwords -- remember the MMX technology commercials? -- and new "generations" of processors. They've even started omitting processor speed, but that confused the public so much that the GHz numbers are now back.
And for a good reason: in most processors, a faster clock speed is better. More cycles get more work done. The processor uses a lot more power and gets plenty hot, but it has fans to cool it down and it's plugged into the wall. Phones and other mobile devices don't have that luxury. They use a battery and they are not supposed to get hot. So simply jacking up the clock cycles isn't a very good idea. But are there alternatives? There are. NeoMagic has one.
NeoMagic's proprietary massively parallel computing technology, APA, is a programmable computing block that runs in parallel with the on-chip ARM 926EJ processor. The APA core does multiple operations for every processor cycle. Which means it uses a lot less power. In fact, NeoMagic proposes to replace the current way of measuring mobile processor performance -- mW per MHz -- with a new one that measures power needed to perform a certain amount of work: MOPS/mw, or millions of operations per second per milliwatt. In addition, the programmability of the NeoMagic MiMagic 6 processor's APA core means it is easily adaptable to a variety of tasks. This chip brings low-power consumption programmable parallel processing to handhelds and phones.
Why is NeoMagic's programmable parallel processing approach so important and revolutionary? Because the multimedia challenges are rapidly becoming overwhelming to most chip foundries. Digital imaging, two-way MPEG-4 video, interactive online gaming, etc., are so processor intensive that the only alternative to very high (and impractical) clock speeds is to add dedicated specialized hardware accelerators or shoehorn a co-processor, such as a DSP, onto the main CPU chip. Those approaches are not programmable, and if scaled to deliver more performance, power consumption again becomes a major issue.
How does NeoMagic's massively parallel APA work? It is based on a "single-instruction multiple-data" (SIMD) architecture that uses 512 data words of 160 elements each per clock cycle. Each of these elements has its own storage and logical processing. To make it all work requires some clever programming. For that, NeoMagic provides a suite of library functions and middleware to, among other multimedia tasks, implement 3D acceleration and video codec processing.
As a result of the MiMagic 6+ APA core's ability to run complex video algorithms, the video subsystem supports four modes: video conference, portable video playing, video camera, and still camera. It passes MPEG-4 tests and can run H.264, can do full-duplex 2-way video, adds video camera post-processing features, and also does 2D/3D graphic acceleration, with 3D acceleration approaching, or surpassing, that of dedicated game consoles of just a few years ago.
But what about the common concern about parallel processing, the fact that a lot of data simply cannot be processed in parallel and needs to be broken down into sequential streams, thus reducing the advantage of parallel processing or rendering it moot? Not the case with multimedia. Multimedia is just about perfect for parallel processing. Which means fewer clock cycles, and less power needed.
NeoMagic's vast patent portfolio -- a portfolio so attractive that techo-giant Sony has licensed it -- includes another neat trick. The company's on-chip embedded DRAM (dynamic random access memory) technology in graphics controllers was the industry's first commercially successful use of embedded DRAM in its graphics controller days, and it helps make the current MiMagic processors more efficient. NeoMagic found ways to merge processing and memory in a manner that eliminates moving data from local cache into the processor and back. A seven-layered concurrent internal bus handles internal traffic. That both lowers power consumption and increases performance. In essence, NeoMagic drew upon its experience and patents to merge computing and memory and thus eliminate bottlenecks. And all of this fits into a tiny 13 x 13 x 1.2 mm package.
And the company isn't standing still. The MiMagic 6+, a superset of the MiMagic 6, sports additional integrated peripherals and further power optimization. There is USB-OTG (USB On-The-Go; a USB 2.0 standard for connecting devices without the need for a host PC), Sony Memory Stick support, an LCD bypass mode and a generally optimized chip layout.
Handling programming complexity
What's not to like? Well, one concern is that the APA uses a somewhat unusual and rather complex programming model, which means a learning curve for device manufacturers using the NeoMagic chips. Initially, NeoMagic supplied a library of functions, but they did not contain all the routines required by a new application. The number of APA tools has grown and now includes fully optimized libraries for most commonly used functionalities.
Still, the question comes up whether the development tools are mature enough to allow programmers to quickly create their own routines. Syed Zaidi, Vice President of Corporate Engineering, says "Now we do have assemblers and C compilers available to program APA," but allows, "Having said that, it is true that APA is a massively SIMD parallel processor engine and optimization needs some experience." "Not a problem," says Jeffrey E. Blanc, the company's VP of Worldwide Sales and Corporate Communications. "Every client has full access to our team of APA programming specialists, and we deploy them to our clients." And while NeoMagic is not a large company, most of its employees are researchers, almost a hundred. Special assistance comes from NeoMagic's Israel facility where, under the leadership of Dr. Avidan Akerib, much of NeoMagic's APA R&D is being conducted. Akerib, a mathematician, has over 20 years of experience in image processing and parallel computing architectures.
NeoMagic's competitive advantage
Given Intel's imposing market presence and the company's success in positioning its chips as the gold standard in almost any computing device, why does NeoMagic not seem concerned about Intel's PXA application processors found in most PDAs and many smartphones? "Intel has always pushed megahertz and mega-bytes," NeoMagic says, "therefore more power consumption and more cost." That won't wash in next generation multimedia cellphones with their H.264/MPEG4/ DiVX/WMV video decoders. NeoMagic's chip runs those decoders very efficiently on its parallel processing arrays. And customers can easily complete mobile digital television (MDTV) solutions. "We can do that at lower megahertz equals lower power and lower cost," says Zaidi.
So if Intel isn't NeoMagic's competition, who is? That would be the likes of Korea's Core Logic with its ARM-based Hera Multimedia Application Processors that use hardwired video codec technologies; Japanese Renesas Technology Corp. which is now in charge of the former Hitachi SuperH CPU core, develops the DSP-favoring SH-Mobile application processors and also presides over the SH-Mobile Consortium; STMicroelectronics' Nomadik family of multimedia processors that use McubeWorks' H.264 software video decoder; Texas Instruments with its hardware-accelerated OMAP application processors, and several others. Generally, decoding and processing is being done in hardware with dedicated accelerators and DSPs. That means extra overhead and extra power. NeoMagic's APA processor, on the other hand, is not only used for standard video decoding, but also for OpenGL-ES 3D operations, image enhancement such as color correction, stabilization, histogram equalization and so on. If anything else is required, it's a software routine away. No need to add more hardware.
Unfortunately, there aren't any standard benchmarks available to directly compare application processors. Alternate methods include running the video codec of choice -- mostly H.264 decoding in this field -- and then check video quality and power consumption. The MM6+ chip decodes H.264 at 30 frames per second on a QVGA screen at an extremely power-efficient clock speed of less than 100 MHz.
Mobile Digital TV solutions
Mobile television may well be the next big handheld revolution and has the potential of revolutionizing TV the way cellphones revolutionized the communications industry. At the 2006 3GSM Congress in Barcelona, show attendees were to witness a mobile TV solution developed by NeoMagic and The Technology Partnership (TTP), a European product and technology development and licensing company whose objective it is to "create new business based on advances in technology." TTP contributed its DAB-IP platform, NeoMagic the MiMagic 6+ processor capable of simultaneously handling H.264 video decoding, audio decoding and high-speed 3D graphics. The demonstrator platform showed live broadcasts from the BT Movio mobile TV service.
Given that there are competing standards (DVB-H, MediaFlo, and T-DMB), flexibility, programmability, and generally being nimble on your feet will be crucial. The prize is a big one. British audiovisual research firm Screen Digest predicts that over 250 million MDTV cellular handsets will be shipped by 2012, with a 43% Asia, 32% US, and 25% Europe split. A NeoMagic powered handset with 30 frames-per-second video, blindingly fast Java apps, console quality 3D performance, and long battery fits beautifully into this scenario. And the company has already demonstrated audio/video processing solutions for the most promising mobile digital TV standards, including ISDB-T (a Japanese digital standard), DMB (already used in Korea, and soon in several European countries), and DVB-H (commercial launches expected this year in Italy and the US).
Here again, the trick will be for NeoMagic to convince some of the Taiwanese OEMs and ODMs to use the chip in their MDTV-enable PDAs, smartphones and other convergent devices. The first examples of such phones are now showing up. Taiwanese PDA and smartphone expert HTC has been showing their "Trilogy" platform, a digital TV and radio phone with a 2.2-inch screen, and a 1.3mp camera.
NeoMagic's answer is the NeoMobileTV product aimed at all global mobile TV standards. This will be available in mid 2006 and its goals are to add mobile digital TV not only to high-end phones, but even those selling in the US$150 range (with the NeoMobileTV part contributing less than US$10 to the bill of materials). What that means is that we'll be able to watch, and listen to, high-quality H.264, Windows Media 9, MPEG-4 or DiVX digital video almost anywhere. For how long? Well, the NeoMobileTV solution promises to use well below 100 mw of power under load. That means that depending on the hardware used, you can watch between four to eight hours.
"Our NeoMobileTV product focuses on the handheld space, mostly cellphones. It has H.264, MPEG4, DiVX, WMV and other video decoders running on APA and we're also providing a complete customized MDTV system software solution. Our customers are seeing better performance at lower megahertz, which means lower power consumption and lower cost," says NeoMagic's Zaidi.
Next-gen GPS devices
Handset-based mobile digital TV is not the only exciting new technology on the horizon. Those who follow the PDA market know that GPS-enabled PDAs have been hot, grabbing an increasing percentage of overall PDA sales. Mitac's Mio line has been quite successful, as have Garmin's Pocket PC and Palm-based GPS handhelds. Mapping, of course, is another application that greatly benefits from a high-performance/low-power architecture. NeoMagic teamed with NordNav and SiGe Semiconductor to create a cost-effective reference platform for feature-rich GPS-enabled multimedia mobile handsets and dedicated personal navigation devices. NordNav makes software receivers that allow quicker lock-on, yet conserve battery power. Quicker lock-on so happens to be a key factor to increased GPS penetration into the gigantic GSM cellphone market. Since GSM is an unsynchronized network technology, conventional GPS systems fail to lock on quickly and may then resort to the very imprecise Cell-ID method. The combination of the SiGe SE8088L receiver which is nearly immune to co-located cellular radio interference, NordNav's E5500 software, and the NeoMagic MM6+ delivers a solution that's capable of continuous real-time measurement at signal levels below -185dBW. Imagine that capability in your tiny next-gen digital TV multimegapixel smartphone that never heats up and has great battery life to boot.
Does it run Windows Mobile?
Affirmative. The NeoMagic MiMagic 6+ board support package has Windows CE 5.0 certification from Microsoft.
The big picture
NeoMagic appears to be perfectly positioned to deliver a key technology that will accelerate mass adoption of mobile digital television on handheld devices. A number of obstacles remain before MDTV handsets become the Must-Have-Gadget of 2007, but these are hurdles more for the content companies and wireless carriers than for device makers. NeoMagic is not the only thoroughbred in this race, but they increasingly look like the favorite. Their massively parallel approach makes all kinds of sense. If properly managed and implemented, the MDTV dream could pop into our lives out of thin air -- like magic.