Archive for June, 2008
Well, we broke down the FBIN file into some sections and validated that the section identifiers were correct by locating the size following the section identifiers. I’m assuming at this point that the section sizes are part of section headers, and I don’t know how long the headers are. Thats all about to change.
I’ve found the FPGA code that is executed on the Xilinx chip on the SD14 and it is possible to reverse engineer it to both a layout and an Verilog file. Let me explain this first milestone in my hack of the Sigma SD14
First off, we’ll examine the largest BINS section in the FBIN file. I’ll work again with the broken down s14v101.fbin file as generated in my previous post. This is the section that we want:
13: 0×001e4508 1983752 0×00076e40 486976
Lets look at the first few bytes as hex and text, and then I’ll explain some things. the first few bytes are:
42 49 4E 53 24 00 00 00 00 FF 30 00 00 00 00 00 BINS$....ÿ0.....
20 00 00 00 04 00 00 00 00 00 00 00 76 B7 80 AC ...........v·.-
00 00 00 00 00 09 0F F0 0F F0 0F F0 0F F0 00 00 ..... .ð.ð.ð.ð..
01 61 00 08 73 64 37 2E 6E 63 64 00 62 00 0B 33 .a..sd7.ncd.b..3
73 32 30 30 74 71 31 34 34 00 63 00 0B 32 30 30 s200tq144.c..200
36 2F 31 31 2F 30 32 00 64 00 09 31 38 3A 30 35 6/11/02.d. 18:05
3A 30 37 00 65 00 01 FF 88 FF FF FF FF AA 99 55 :07.e..ÿ.ÿÿÿÿª.U
66 30 00 80 01 00 00 00 07 30 01 60 01 00 00 00 f0.......0.`....
The green values are the hexadecimal, and the blue values are the strings. Now, we can see the BINS tag starts things off, and checking the size…wait a minute! the size is wrong! What? Well, then, what comes next? To find out, we had better figure out whatever sd7.ncd is. SD7?? Yup. there is more than one reference to the SD7 in the overall firmware file
Lets see if we can find anything else in that string that will help us. How about 3S200. Hm, the Xilinx chip is a XC3S200. Jackpot! A quick google confirms. NCD files are Xilinx design files. That gets us somewhere. I have the feeling that this is the file that the FPGA loads as the design. Another quick google turns up a page on FPGA-Faq.com that is entitled “Tell me about bit files.” Bit files are the FPGA’s firmware file, quoting from FPGA-Faq.com:
The Xilinx .bit format is pretty simple. It uses keys and lengths to
divide the file.
Here is an example. Below is a hex dump from the beginning of a .bit file:
00000000: 00 09 0f f0 0f f0 0f f0 0f f0 00 00 01 61 00 0a *………….a..*
00000010: 78 66 6f 72 6d 2e 6e 63 64 00 62 00 0c 76 31 30 *xform.ncd.b..v10*
00000020: 30 30 65 66 67 38 36 30 00 63 00 0b 32 30 30 31 *00efg860.c..2001*
00000030: 2f 30 38 2f 31 30 00 64 00 09 30 36 3a 35 35 3a */08/10.d..06:55:*
00000040: 30 34 00 65 00 0c 28 18 ff ff ff ff aa 99 55 66 *04.e..(…….Uf*
Oh, this looks just like the section that was found in the SD14 firmware file! Reading more at FPGA-FAQ.com, we see that there should be an “a”, “b”, “c”, “d”, and “e” sections, and sure enough, our “a” identifier is at offset 0X076e72 in s14v101.fbin. You can easily find the rest of the sections yourself.
We just found the FPGA code that runs on the Sigma SD14!
Googling turns up a bunch of info for these Xilinx Bin files. Here is a program to decode the headers. Here is some info on how this can be converted into a C header file. Good thing Sigma is cheap and didn’t use the more advanced 3A FPGA or we’d have to deal with this crap.
The greatest piece of info I found however was this brilliant site on a debit utility to turn a BIN file into a netlist. For those of you who don’t know, a netlist is the list of circuit elements that make up the circuit. In this case, it takes the bin file and generates a list of logic gates for the circuit. This will also give a visual representation of the circuit, and even contains information on how to convert this back into verilog. Verilog is very easy to simulate (I’d imagine that a Xilinx bin file is as well)
Next steps: Convert the SD14 Xilinx bin file into a netlist, generate a graphical representation of the circuit, and convert it back to verilog.
So a quick recap was dumping the largest CMbT section from the s14v101.bin file. I removed what I thought to be the header (everything before FBIN). I renamed this file s14v101.fbin.
Now, I’ll use the parsebin script to dump all the BINS sections. How do I know to do this? I took a look at the file and found a handful of occurrences of the string “BINS”.
So, the steps, create a new sections.txt file that contains one section: “BINS”. Call it fbinsections.txt. Now, run this through the parsebin.pl file like this:
./parsebin.pl s14v101.fbin fbinsec.txt > info-fbin-101.txt
This gets us the follwing information:
Num Hex Length Dec Length Hex Position Dec Position
0: 0x00000a4c 2636 0x00000018 24
1: 0x000002a0 672 0x00000a64 2660
2: 0x000001a0 416 0x00000d04 3332
3: 0x00063c54 408660 0x00000ea4 3748
4: 0x00003db0 15792 0x00064af8 412408
5: 0x00000fc0 4032 0x000688a8 428200
6: 0x00000024 36 0x00069868 432232
7: 0x000002f4 756 0x0006988c 432268
8: 0x00000034 52 0x00069b80 433024
9: 0x00000230 560 0x00069bb4 433076
10: 0x000001a4 420 0x00069de4 433636
11: 0x0000bef8 48888 0x00069f88 434056
12: 0x00000fc0 4032 0x00075e80 482944
13: 0x001e4508 1983752 0x00076e40 486976
Interesting. 14 sections of greatly varying length. A sanity check right about now would be a good idea. Lets see if we can make any sense of this quickly. We can examine the BINS 0×069b80, dumping the hex:
42 49 4E 53 34 00 00 00 00 03 40 FF 00 00 00 00 .....
It continues, but I chose to leave the rest off. The text in red is “BINS”. According to parsebins.pl, the hexadecimal length of this section is 0×34. Hey, there is the length!! I’ll leave it as a homework assignment to verify the rest of the sections. Looks like we’ve found an embedded file type, that has at least some structure. What are all the sections? Whats that really big BINS section? We’ll leave that for next time.
So, as I mentioned in the last post, there were some large sections in the bin files that require further investigation. The output of the parsebin script shows these sections. The most interesting to me is the large CMbT section. Lets look at s14v101.bin first. from parsebin:
Num Hex Length Dec Length Hex Position Dec Position
0: 0x00000040 64 0x000001d8 472
1: 0x00000040 64 0x00048b54 297812
2: 0x00000044 68 0x00048bbc 297916
3: 0x0025b318 2470680 0x00048c30 298032
4: 0x0000004c 76 0x002a3fe0 2768864
5: 0x000000f4 244 0x002a402c 2768940
Hm. that 4th CMbT section is the one I want. Using parsebin to dump this section to its own file:
./parsebin.pl -p CMbT -d 298032 s14v101.bin
This will generate a file called “s14v101.bin-CMbT-298032.” Lets open this. The section starts off like this (plane text):
CMbT....L.......(...ColorspaceTestText......Colorspace is sRGB-Enhanced!....FBIN.........O®±....dn..BINSL
.... ÿ.... ...,..8+.Õ
Woa. Whats that? We see a couple possible tags. There is that FBIN tag. then BINSL? Hm. FBIN=Firmware Bin?? I have the suspicion that there is a header associated with the original CMbT section, then there is an embedded file, an “FBIN” file. Searching this file for BINS and I get several hits. Letsuse parsebin to break this up too. First thing first though, lets remove everything before “FBIN” and save this as its own file. I’ll call this file s14v101.fbin.
So Sigma released an updated DP1 firmware today. Couple this with some interesting shots of the insides of the DP1 and turns out that there is a Xilinx chip and a custom ASIC (the Sigma TRUE) on the DP1. Way to go Sigma, custom ASICs, nice. According to the DP1 firmware, the Xilinx chip is a XC3s200 - same as the SD14. The firmware is also made with the same CamXML2Bin program, looks like Foveon release this? Don’t know.
You might wonder how I was able to determine the Xilinx chip from the firmware. We’ll, I’ve made incredible progress in the last couple days, I’ll try to write about it tonight.
I love perl. Its great for parsing files, even binary files. The ParseBin.zip zip file contains a perl script I wrote (releasing under GPL 2.0).
The script file basically attempts to parse the binary file, looking for sections. It will keep track of section location and size, print out information on the section, print individual section types and even dump or ignore sections at a given location. The command line arguments to the script are:
./parsebin.pl [-h -s [-i Location] [-p Section] [-d Location]] BINFILE SECFILE
-s: Print Strings as well
-i [Location]: Ignore (dont print) the section at [Location]. Can be used multiple times.
-p [Section]: print a section. ALL for all. Can be used multiple times
-d [Location]: Dump the section at location to a new file.
Its not pretty but it works for now. This will evolve to allow individual sections to be diffed between different firmware versions. For now, its a tool to investigate the firmware sections. The sections.txt file I use contains CMbP, CMbT, CMbM and CMbH, each on their own line.
The Section Info zip file contains the dumps of the script when ran against the s14v101.bin and s14v102.bin files. The sections look pretty smiliar. There are a couple large sections that I’ll want to look at closer. The -d option to the script will help here.
Lets take a look at the strings in the ver 1.01 SD14 firmware file.
We can dump the strings using the strings program:
$ strings s14v101.bin > s14v101.strings.txt
This dumps all the strings in the s14v101.bin file to the s14v101.strings.txt file. Open that text file. You see right near the top:
Converted using Foveon CamXML2Bin 22.214.171.12424 Release (Mar 20 2007 16:31:42)
This is located at offset 0×00016C in the original file. Looks like they use some tool to take an XML file and convert it to a bin file. Googling this name does no good, it looks to be an internal proprietary tool.
Directly below that we see the text “CMbH”. In two more lines, we see “CMbT”, then “CMbM.” You can grep the strings file for all occurrences of CMb*
$ grep “^CMb.” s14v101.strings.txt > sections.txt
Taking a look at this, we can see that there are four unique CMb* sections:
I’m thinking that these are the section identifiers we are looking for. the next step would be to break the binary file up based on these sections and to collect some basic data on these sections. Spend some time looking into the strings, these sections may not be the only ones.
We can grep for all the 4 character stings that are on one line in the strings file. Lets try this:
$ grep “^….$” s14v101.strings.txt > sections.txt
There are lots of mishits here. a quick glance makes me want to look into the following more closely:
Also, of interest from the strings file:
Hey, those are the sections in the X3F files And:
Update MCDSP bootstrap code from CF.
Update BFDSP bootstrap code from CF.
Reprogram FPGA code from CF.
Update CAMCPU firmware from CF.
Looks like code for all the processors can be updated via the CF card! Awesome!
Starting debug console. (type HELP for help)
I want access to this debug console. The software fror it appears to be included in te firmware. There is lots you can do with this, just take a look at all he descriptions for the possible commands!
The next step will be to break down the firmware into sections based on the CMb* tags. I’ll also take a closer look at the other sections.
After spending an evening poking around the ver 1.01 firmware, I came up with a plan of action. It appears that the firmware has areas of strings that is human readable, areas of code that is not, areas of padding (i.e. repeated patterns that done seem mean much at first glance). We need a systematical way of analyzing the firmware files.
I once put together my own decoder for the X3F files and I recall that the file was broken down into sections. The sections were all labeled (i.e. FOVb, SECd, ect). Perhaps this file is the same. This seems to be in line with first observations that there are string sections and data sections. This also somewhat aligns with some other executable file formats.
My plan is to:
- Dump the strings for a firmware version and look for hints.
- Look for section identifiers in the firmware.
- Attempt to break the firmware down into sections
- Compare various firmware versions to each other based on sections.
The approach will use the first few versions of firmware for investigation/comparison. I’ll leave a couple versions, version 1.06 and the two 1.07s, uninvestigated as an unknown baseline to test my final product.
So, as I pointed out in the last post, you can find great things just by poking around the firmware with the hex editor looks at strings. I’m looking to find out what hardware the SD14 has in it. I’ve heard reports on DPreview and elsewhere that the SD14 has a Blackfin DSP. Lets see if I can corroborate that.
At Offset 0×093993 we find:
THe BF561 manual notes that a larger lock count may be necessary when changing the voltage significantly
Complete with typo and all. I thought those Sigma guys were Japanese, what are they doing typing their notes in English? Perhaps its easier. Perhaps there is another reason. That would be a reference to the Blackfin BF 561 DSP. The DSP is a dual core chip with a “High data throughput tailored for the needs of imaging and consumer multimedia applications.” Looks like we found our chip.
The Blackfin site has links for all the technical documentation and even development evironments for the BF561. Get reading, this is the brains behind the camera. But is it the only chip?
Not quite. Looking near the end of the file (offset 0×0DFAC8), we see the following:
SDK for DSC: Copyright (c) 2003 MegaChips Corporation
Further down, we that perhaps this is called the DSC 131. You can also see references to an FPGA throughout the file. Neither of these chips are FPGAs, so there is at least another chip in there we don’t have an ID for.
Lets go Googling. This page from Planet Analog is interesting. It tells us that the SD14 has inside:
A couple observations from this:
- The FPGA must be loaded with some sort of executable. Can they update this?
- This camera has 2.5Gb of RAM!
- What is run on the MegaChips chip?
- Whats run on the Blackfin?
- How is it all coordinated? I’d suspect that there is some form of OS on this camera.
- How is all of this “booted?”
At least the observations from the firmware are confirmed
So, there will be a flurry of activity at first for this firmware hacking project of mine. Here are a couple of the first steps I took and a quick brain dump of my first thoughts on this.
Sigma has released 8 versions of the SD14 firmware so far Ver 1.01 -> ver1.07. Ver 1.07 actually has two versions with it, 1.07_0 and 1.07_1. Once upon a time you could download all of them from Sigma’s website, though it looks like the early ones are not available any more. Good thing I kept a copy laying around.
The first and easiest thing to do is to open them with a hex editor and see whats inside. Perhaps we’ll get lucky and something will be readable. I used Okteta or KHexEdit, depending on which machine I’m on (I use linux, Fedora usually)
Sure enough, the folks at Sigma were kind enough to leave much of the file readable. There is a wealth of debug information in the files, for example, s14v101.bin opens with the following text:
Hm, looks like this is Sigma Co’s SD14 firmware version 1.01 made on 04-11-2007 Easy enough. Surprising, a lot of information can be gathered just by looking at the firmware like this. as I noted here, the firmware contains a lot of debug information, in particular I found this to be interesting:
Specify which steps to skip in the Image Processing pipeline.
Skip AutoAdjust = 0x00000001
Skip LinLUT = 0x00000002
Skip ColumnFilter = 0x00000004
Skip ColumnFilterSegCapacitance = 0x00000008
Skip SpatialGain = 0x00000010
Skip ISOGain = 0x00000020
Skip DriftCorrection = 0x00000040
Skip FlareCorrection = 0x00000080
Skip SatDetect = 0x00000100
Skip BadPix = 0x00000200
Skip RedSharp = 0x00000400
Skip NeutHilitePixel = 0x00000800
Skip NeutHiliteNeighbors = 0x00001000
Skip NeutHiliteSmoothNeighbors = 0x00002000
Skip Despeckle = 0x00004000
Skip SpatialDQ = 0x00008000
Skip ChromaBlur = 0x00010000
Skip ColorDQ = 0x00020000
Skip CorrectionMatrix = 0x00040000
Skip ToneGamma = 0x00080000
Skip Sharpen = 0x00100000
These appear to be the descrete steps in the image processing pipeline. You can find this at offset 0×0AA846 in s14101.bin. Boy, I wonder what algorithms are going on behind the scenes there
So, as I said, much can be learned just by opening up the firmware in a hex editor and poking around.Thats what I’ll start off with. Open the firmware and see if there is anything I can pick out.
Sigma released their SD14 camera a while ago and as I have an SD10, I don’t see a very compelling reason to upgrade. The feeling I got from the camera was that it was underwhelming, the Photo Pro software original released was horrible, and then I started to see an endless release of firmware version after firmware version to correct bugs.
Now, of course many of these issues have been fixed, the Photo Pro software is now based off the excellent 2.x version as opposed to 3.0 (I’ve got some details on the company behind the 3.0 mess), and the firmware releases have seemed to subside. Perhaps they are now getting the bugs worked out of the camera.
The firmware however intrigued me, never before had sigma done this, released so many firmware versions for a camera. Recalling from memory, the SD9 had 1 update and the SD10 didn’t have any. If I have this wrong I’m sure someone will be so kind as to correct me. With all these versions of firmware sitting around out there, I got to thinking, I wonder whats in there.
It dawned on me that even though I have an engineering background, and at least some experience with embedded device programming (an old school project on a PPC 405GP) I really have no idea how to reverse engineer firmware.
You hear about it all the time, these l33t h4×0r$ reverse engineering DVD encryption, great work has been done hacking the low end Canon Powershots, and even phones getting a workover.
How do these people do it? How is the firmware decoded, how do they reverse engineer it? What does the firmware even look like? I decided that since the nice folks at Sigma gave us so many firmware versions, and even an excellent changelog between them, I figure that will a little time and effort, I could probably at least figure out what the firmware looks like, how to decode it, and hopefully how to change it and re-encode it. This would motivate me to buy an SD14!
Why? Because I’m curious. I like to make my own things. I like to understand how something works from the ground up. There are some features I want, and I don’t particularly think that Sigma is going to listen to me much (I tried once - didn’t happen) . I also think there is a good chance that I can actually do a better job.
So, what I’m going to do here is chronicle my adventure in attempting to become some elite firmware hacker. I’ll tell you all I know, and maybe, if anyone is paying attention, someone else might find it interesting.