Interfacing a cheap phone camera module to a PIC32 microcontroller.



Views:124339|Rating:4.86|View Time:26:8Minutes|Likes:1533|Dislikes:45

I've recently done this 500 mil circular LED display for a LED manufacturer it's got white and blue LEDs on there and we thought it'd be quite fun instead of just playing back and fixed content to sneak a camera in it so I'm right in the middle of here we've got one these little mobile phone camera modules so I just wanted to talk about actually interfacing one of these toward Michael control all that needing to go to FPGA to produce a nice simple way of getting a fairly low res output this is basic 64 by 64 and to get a fairly low res output to it basically a serial stream to feed into low res displays so as well as the camera input this will actually display often SD card I'll see that I see right these cameras there yeah they're small enough you can actually get this in the sensor and this made out for PCB so there's a little gap in the center by what I wanted I didn't have a great deal of time to get together get this running so I basically wanted a fairly simple solution to just get that camera input to a la res serial stream but this is actually running a 2 megabits which is just about enough to maintain 25 frames a second at that Padre I want to keep the baud rate reason me low because we might want a little bit cable on it a few years ago I did something vaguely similar with these panels and these are very low res panels and for that I used this this is a fairly low res say at CIF resolution field monochrome sensor which you got anymore and this was using an arm 7 processor with some very carefully crafted f IQ drop code we've had to slow the frame of the frame rate on this was a little bit on the low side so I wanted to take a fresh look at doing this with a more modern camera module they made most you know play around these cameras before this was driving into one of these ipod nano displays but because of the high data rate this needed an FPGA now for this project basically the requirements were different one was that I only wanted monochrome and the other one was I only wanted about 64 by 64 pixels and so more under the bit more flexibility of being on a microcontroller so I thought have a look see what was possible on just purely on the software software only basis the interface on pretty much all these camera modules is the sign Lisa that'll parallel interface one some of the more modern ones use a MIDI interface but most the other the older ones I've got pretty much identical interface this particular one is a no V 965 oh it goes up to SX VGA 1280 by 1024 will be a fairly low framerate there's quite a number of these available really cheaply as modules and one nice thing about the 9 650 is one the best the one for which most of the documentation has escaped it's quite hard to get foot detailing for a lot of these you can graph a lot a datasheet but that doesn't give you important things like all the original it's the setups but there's about three different documents available for the 965 own it's also really cheap and I bought it's a mobile if you try over 70 of these cameras for $70 so they're dirt cheap really another fairly common one is the IV seven six seven though which i think is goes out to VGA again it is very similar the main difference is the the I frequency registers to tend to be different they need the different set of registers to initialize them but basically the interface is pretty much the same you've got basically three timing signals you've got vertical sync which is what goes low ones per frame you've got a horizontal sync well it's really speaking it's horizontal reference which tells you where it where the active pixels or they tend to call it href rather than age thing but it's pretty much the same thing you then got a pixel clock which is a high frequency clock that that then clocks the day to out this problem this is one of the the data lines so if we look at a whole frame if a fan white liner from the camry prejudiced fancy there's some changes going on on there so in addition to this unit to supply it with a clock which is typically 10 12 or 24 megahertz someone have a PLL on board so you can feed it like 12 makers and generate 248 Freire makers internally now I there's two problems interfacing these things for my controller firstly is the data right you know these things are spitting out date or even at QVGA it's spitting out nearly item it's spitting out seven point six megabytes per second but also the amount of data yet most micros don't have enough memory to hold on full frame now for I want to do the the memory issue isn't a major problem but the date rate is it is possible to just slow the pixel clock on these down to write which you know you can actually read it this is coming out the prom is also reduces the frame rate I haven't come across any camera module we can sit into a low resolution and a high frame rate without having to accept all the data at the speed is K is coming out now one advantage I've got is that I'm only interested in monochrome typically what happened what you get with it each pixel clock you generally get two clocks per pixel it's take your Suzy an 8-bit data bus and there's two common formats one is y UV which is intensity plus component that's typically used for when it's being JPEG encoded so what you generally get is y uy v UI u VAR by V now for practical applications we can just use that Y which is the Luminess and ignore the white the UV so one advantage of that is that your you know you can only look at alternate pixels which immediately has your data right the other common format is an RGB 565 format now the problem with that is that you need to take each pixel pair because you know you've got this overlap between the red and red green and blue bits over those two clocks and although you could let's say take every fourth one the problem is that you have say you know the red bits from um picks on the blue bit from one further across which although would look generally ok you'd get nasty fringe ly twinkly bits at the corners where you know the bits were different between the two pixels now I'm implementing this on our pic 32 series device I use quite a lot recently unlike Wiley elope they've got quite nice set of features for the views and quite a number of projects one the features in particular which is useful for this is DMA and which is direct memory access if you're not familiar with it basically what DMA does is on some triggered event which is typically the same as an entry at pretty much in the case of the pic32 anything that can create an interrupt can also trigger a DMA and what the DMA does is it will initiate the transfer of one or more bytes of memory either from memory to a peripheral or from a peripheral to memory and you can also usually do memory to memory as well block transfers so in the case of let's say for example reading data from a serial port instead of having a serial interrupt that handles that data it can actually fairly be handled in hardware which basically means you've got a much lower latency high throughput but also it's not disturbing your main yeah it's not it's not using up any significant amount of prizes the time it does it does use some memory bus time but it's trending it's generally a lot faster so basically the method we're using to actually get the de trainers we do do using DMA now there's two ways of doing this one is we can just using a bit parallel import port so we set the DMA up to read that port and then write it to a memory but they pick 32 is also parallel math support which basically emulates a microprocessor style bus which gives you external chip select read and write signals now one advantage of using that is that you actually get an external read signal which you can actually use to look at the timings so you can know exactly when it's reading the signal from the bus which can be handy especially when you're really pushing the speeds this is the one bit of the camera light cameras they treat output now although this appears lucky tutoring it's actually the other one because we're triggering on the read pulse what's actually happening is that some repulse is sometimes getting delayed by sort of parts of a clock pixel clock and that's because of the the memory process in the pic in that although the DMA process is much faster than the interrupt you still sometimes get a very small amount of latency if the memory is busy when the DMA comes in it can delay it by cycle auto and particularly if you're in long line you still monitoring by about 2 or 3 processor clocks but one handy thing about having this is you can actually see how much margin you've got so you can see it's actually doing the read on this vertical edge here so you can see we've got a nice clean page of sample if I just send a command to the camera to invert the clock phase we'll see that it was going to start it's now starting to occasionally sample at the edge and you can see on the picture that we're getting garbage data on the occasions that it's something that Rompin in that wrong place so yeah that that's one nice advantage using that panel most of what you've actually got this marker that tells you exactly when it's reading so nothing else it really helps with debugging making sure that you know your your counter facing is right to get actually sample data when you think of sampling it now one important thing that you can usually configure on the camera modules this pixel clock although this occurs even outside when you're getting data you can configure it to not occur during the aging period by default normally it just occurs all the time but you can actually can configure it for I squared C to not be clocking during this icing period and that that's actually really important the reason that's important is that you need to get consistent phasing between when you're actually reading the later in and the pixel clock so if that pixel clock is running when you're trying to setting it setup timers and stuff then it's going to get out of phase so basically what we do is we take the pixel clock and then we feed that into a timer and that time what I reload so for example if we want to get every pixel we set this reload value to two so what happens it'll count up 0 1 then when it is 2 it will reload the timer and generate the DMA trigger so by just setting this reload value we can tell it how many pixel clocks per diem I operations so for example for every 1 pixel which is 2 pixel clock we load that with value 2 so it goes all 0 1 0 1 0 1 but for example we could also sample let's say every third pixel every fourth pixel purely by setting that and on our basically on our vertical sync in drop we'd you've reset everything for your start of a new frame then on the horizontal sync we setup for each light for each line that we're going to transfer so which for example we initialize this time or two to zero so that we know that our clock phasing is going to be right the reload value will just initialize at the start of the frame because that's never going to change the other thing we do H sync is any lines keeping some see if we're doing a vertical any vertical scaling we just ignore lines so we just have a little counter so we only enable and start with DMA process on the lines we're interested in the other thing we do is obviously we need to know where about on the line we want to start so again we use this timer this timer so all those who said that sort of reload what we effector it is we set a negative value into this timer so let's say for example we want to start to start a hundred clock sweet during the h sync we load the time with minus 100 so it'll count ups or through then when it hits zero it will then start the DMA and then reload so it goes from minus 100 to 0 and then 0 1 0 0 1 etc along the line and one thing is important is that this I think interrupt because here are the beginning the like that the sinc post will see the interrupts will take a little time to actually happen and for us to do what we need to do so all that happens here while the pixel clock is not running so that's all sour print runs finish so we're just ready to as soon as that pixel clock starts that's going to start that timer gets to the point where you want to start sampling and then to do a pixel clock sampling and the DMA controller has a value that tells it how many bytes to transfer so that's what stops the process at the end of the line of how many so let's say we want 64 pixels of line we said we tell they're making trailer to transfer 64 bytes and we can also get it to generate an interrupt after it's done that if we need to which can be useful but it's not actually essential what we're doing and the other thing is we're generating the clock for the camera module we could do this for another timer with a an output compare but on the pic32 it's got this ref clock out which basically is a simply divided take the cpu clock and divided by an integer the advantage you're using that over timers this will actually do a divided by one so we can actually feed the full CPU clock forty eight megahertz out for the clock input now I don't think these modules will actually take 48 makers but if you're say running everything at slower clock that could be an advantage that just gives you a little bit more flexibility in your clock division ratios it's also important to actually clock the camera module from the process of rather than giving its own independent clock again to make sure that everything stays exactly locked together in in the right phasing because you're you're trying to hit quite a narrow window when you're solving these pixels even if you only pick something relatively slowly you've still got actually sample within the window of I am a single pixel clock time so if you're going to pendant clocks you know they're not going to stay in phase accurately enough that happens so it's very importantly you actually clock the manual the either clock the camera from the pixi internal clock will use a common clock to drive both of them but so the pic32 is flexibly enough that you can generate a reasonable range of clocks using its internal PLL then divided is required and so that a lot of care and modules have got PLL as well so if you need a high-low say 48 megahertz clock here you can do that one if you you quite and run into is that the inputs to these as tens have a latch to synchronize them so if your pixel clock is of the same as your cpu clock you may find you can't actually get it to clock fast enough because it needs to Yoshi has he'll need at least one CPU clock sluggish transition so if you're if you're really trying to push the boundaries of speed things can get quite messy though and you may get it to work but it's going to be a bit of trial and error and so we can see here these are this at the point where actually slamming the image this is one complete frame at the moment and you can see the image on the right so it's a fairly zoomed in so by instead of solving every pixel Arkansas several times every – every three etc so you can see the sampling spreading out and also the image zooming out and also by adjusting the delays we can just sample different parts of the frame so I can just adjust the vertical position and on the horizontal this is now showing a single line again we can set the sample position anything scroll from side to side and the zoom factor again we just sampling so every typically every every pixel every two pixels etc to get different so horizontal zoom factors now once you get particular when your zoot you're zoomed quite a long way out because you're sort of sampling small pixels over quite large area you get this is in effect so you see that small objects perfectly fill a whole pixel you can see B value seeing on this of the WindowBlinds over there it's all looks a little bits or sparkly and crunchy and one way around that if you're only ever going to be using one zoom level you can actually effectively apply some analog filtering to this by just D focusing the lens most of these modules they tend to glue the lenses but usually you can actually solve with a bit care release it's a pair of pliers or something just to once the blue bright you can then usually screw them in and out so if I just take this lens out a little bit and also D focusable now see we're actually not getting any of that sparkly thing going on we've got I'll see with the low res it doesn't matter too much if we're out focused because we don't have the detail in the final image but now you see we don't get that Ilia Singh effect as we affect you we effectively added a an optical low-pass filter to it so we've got a relatively sort of clean image without those sort of sparkly aliasing type artefacts right so this is how the the module ended up dish by prototype one that's got lots of wires just I can plug onto it to probe it missing I'm just displaying this on the pieces being fed into FTDI high-speed cables they're the standard 80232 will only run a three mega board with high speed one will go up to six they'll do the other eight five four mega board and so on so this is using high speeds the c-23 to HD cable and that's just being so pimp out and directly from that it's some just encoded some code in here just you said it come on says give me an image and give you the parameter and it just before the data back out you so that's basically the final version of it one thing which isn't very clear if you read the data sheets you have to read in quite a lot of detail to find it if you look at the spec it basically says you you but you've got three power supply pins on a you've got one point a volt CoreLogic two point eight volts analog and I've Iost which is I think one to beat one point eight two three point three to match your whatever you're driving with now certainly the ivy 9650 and five five I don't know about the others you need to look very carefully the datasheet actually have an internal core voltage regulator and if your i/o supplies above I think typically about two and a half you can actually use that so you don't actually need an external one when a volt regulator but you do need to enable that with an I squared C command because I wanted to do like a self-contained camera module I could run everything at the away voltage mostly it's all I've gone this board is a single to point a Vaughn regulator so the pic is powered by two point eight the camera and log and logic supply running on two point eight and it's using the internal regulators one point eight so literally all we have on this board is the pic the suit 1/8 hole regulator and some passes and that's it nothing else on here now the actual peak I'm using is a 32 M X 170 F 256 B this is a relatively new device that the 32 M x1 tears have been around for quite a while but they fairly recently produced the 170 version and the main difference in this has got 64 K of RAM now that's unusually large particulate air 28 pin device to actually have that much ramen of course yeah that's also a very useful thing to have if you're putting images in you want to be able to capture as much image as you can if you're producing a general-purpose module obviously if you know you're only going to one every one 32 by 32 you can use a liar end version the chip that's there's a bit cheaper but the nice thing about this chip is available in you know dip SOS SOP and qfn it's quite cheap like a couple of quid or so but it's got that 64 gave round more DMA and stuff so that's not basically a complete be a serial quite flexible serial camera module but monochrome only now in terms of performance it's actually not bad I've not really spent too much time playing with sort of squeezing the absolute maximum performance out of this thing but because we're doing that you're only sampling alternate pixel clocks to just get the lumens we can actually get the cameras basically running in QVGA mode most of these cameras do have a few different modes so you can program them via I squared C so for example you put it in Q VGA mode it has the pixel clock frequency I new cube you can probably also use that technique to slow down the clock because some them support resolutions below QV 8 VGA but say the fact that you might actually be able to get color out from that by you know sampling every single clock at a sort of moderate frame rate and again I've not really done done too much testing that this will just about get all 320 pixels of Q VGA the main limit of course is how much memory we've got in that pic and some just a bit more symmetrical that's 160 by 120 so that's hard actually a quarter of Q VGA so that's that works quite nicely and so there probably is some scope I have actually I dropped this running at 50 frames a second but one the first problem you hear in terms of speed is if you try to pull that DMA trigger that DMA to quickly what happens is in Steve instead of trickling when you wanted to it's it's always continuously DMA as fast as it can and now lend up being slower than my way you want it so what happen is you'll start getting sampling not actually sampling all the pixels you get you'll get pixels missed out as you change the zoom ratio you get to a point where it doesn't actually get any better because instead of delaying when the timer says it's just the aiming as fast as it can and the data ends up not being probably synchronized to the timer but you know there's quite a lot of scope for experimentation there's quite a lot scope for tweaking the camera registers to put it into different modes changing some delays changing clock phasing and you know there's all sorts of stuff that you can twiddle but you know it is pretty flexible even in a failure to pay for its implementation there are a few other things you can do in software to optimize a performance and overall throughput again you're just taking the general case of grabbing an image and just chucking out the serial port where you could be throwing it to a display or maybe a radio link the first thing is you know what you don't need to do is you know you start grabbing from you got a V your vsync you sampled it you don't need to wait until you've got a whole frame before you start sending it all you need to know is that you've got at least one line so if we look at what's happening here it's sending this is just a header byte and I it's using the DMA complete interrupt to say you know the DMA on that line is complete and we've got a foreground task that just monitoring which line is currently on and making a decision as to any concern so what's happening here is that both delivery slowed the board right down a little bit here so as soon as it's got the first line in you can actually start sending it now that the image browser on this is actually it takes about twice as long to send it it does to capture it but there can be various other scenarios so for example at certain other zoom levels like that for example the actual send time is pretty much the same as the acquisition time so in some cases you don't even need a frame buffer at all in memory few implemented this instead of as a frame buffer as a FIFO you could for example capture depending it on the exact permutation of both the zoom level and the output resolution you could actually end up being able to output quite a lot more of a frame than you actually have memory to store because you don't need to actually store the entire frame in memory all at once and again I mean if you're not changing any of the parameters you don't need to worry too much about v6 so for example here we're capturing quite near the end of the frame we're sort of well people starting to send but it's actually taking us into the next frame time to finish sending but that doesn't matter because we don't really care what happens until the start of our next transfer here so you know we don't actually have to wait for the to wait for a vsync we just need to wait for the start of the transfer the exception being if we change some of the parameters then we do need to wait for freezing because that's the point we need to latch our you know our line positions we might need to for example up the update on the camera I squared see registers but mostly things like our you know all added timer delays and some we would tend to latch those on vsync because in particular vertical position yeah we need to know that to know how many lines to count before we start but for continuously acquiring with the same parameters you know all we need to do is make sure that we send it by the time the next one comes along to avoid needing to drop any frames so you maintain the maximum frame throughput now of course I've done this on a pic because it's why familiar with quite a lot of micro these days have DMA so you know I'm sure it'd be possible to do this on quite a number of other micros arms yet very some prices or whatever yeah you can do it with interrupts the problem is that the interrupt latency sufficiently slow that probably to end up having to slow the pixel clock down a lot and also probably slot slow the frame rate down a lot but also if you don't care about frame rate that might not be an issue but so soon as you have DMA you've got that method of getting data in quickly and if you also have the ability to divide that pigs on lockdown then that means you don't have to you can immediately slow the data down significantly particularly if you don't need color and so if you do then at the very least you need to be able to grab pixel pairs and there might be ways of doing that I mean for example there might be ways for exampie perhaps had an external latch and run into a 16-bit port and there's probably you know again there might be some processes that have facilities to do sort to quick DMA cycles and then a gap depending on the memory architecture so bike worth having a look at that whatever your favorite processor is but the lycée about this pick it's got you know it's got the thirties using a fairly simple sensible package let's go on to your boss later so you know literally you don't need it any significant amount of external components so this is a few these camera modules around also these breakout boards are quite readily available on the epa and i highly express these have got the two point nine one point eight volt regulator on them the this pin header is actually a fairly standard pin out i think he originally came from some omni vision evaluation modules when you find the pin outs in those data sheets the only one that i've found there are different these ones which you've got son yun on these are nine six five five modules in the pin and this is completely different and and if you do find my module it's got a different pin out I mean these are so cheap you know you just buy one if you do firearm actually come across might one's got a weird pin out that you're not sure off you know these are generally so cheaply you can usually sacrifice one just to trace the pins out use of cracked lens holder off and then either just cut around the solder the chip which will be a it's a BGA on there and just medial if you check age pin with the meter the BGA PMF generally in the datasheet so nice about these they do come with the lens when I hope you're pretty much stuck with that lens if you want different different lens what often is you can just so you either screw the original lens all the way out you can you the they tend to be glued in because you crack that low and then they're quite easy to adjust and either leaving like that will just take lens hold off completely and then use an external lens like a standard n12 lens and they'll see there's all sorts of different viewing angles you can get various things including as a few zoom lenses as well you can get in this format so you know it's nice to have it with the lens please and all that difficult to use it with a different end if you really need to

34 thoughts on “Interfacing a cheap phone camera module to a PIC32 microcontroller.

  1. Is there a video where you go over how you made the circular LCD display? And also how you achieve the variable "per pixel" brightness?

  2. hey I'll have an idea that you can make you can make a suit out of screens and cameras so that the cameras in the back of the suit puts the image behind you in front of you and the cameras in front of you puts the image behind you so you make a invisible suit if you read this comment me back so me and you can get credit for it also we can be partners in making a invisibility suit i also have an idea of what kind of screens that will help you with the curve of the suit

  3. Wow thanks for being probably the only person on youtube to go to this depth about how image sensors work! Amazing information!! I've watched this about 12 times, initially knew nothing about electronics at all, done a heap of googling in between, but now understand everything of what your are talking about.

    At first I wondered how a "Piano" fits in with electronics, but now I do know what a "PLL" actually is! I feel confident enough to try and design/build my very first ever electronic circuit using one of the optical LVDS variants .. this'll be fun ..

  4. is there a way i can use this smartphone cams from old samsungs to reuse for some sort of arduino base mcu ? i have not come across any tutorials or breakdowns, and tho you was giving out a lot of useful information im left in the dark as i dont have your technical background to understand half of what you was saying, but i did hear something about 6mb per pixel? or raw data thru the serial ? and most mcu dont have that type of memory ? maybe you can expand on this for me please Mike? im  not looking for High RES im simply looking to find edges and blobs using OpenCV for Processing and my trusty arduino UNO or NANO but my background thus far is in C# & C++ so i tend to stick to this mcu since they understand my lingo or what not but this is for a humanoid application and im simply trying to turn those small cmos cams into eyes for my androids face and i just dont want to get a webcam and tare it down just to implement into the size factor im looking for you know what i mean ? well Mike hope to hear from you !

  5. Is it possible to extend rear 13 mega pixel camera module of a smartphone , by connecting it by a longer ribbon cable to the phone? So that it acts as a normal rear camera?

  6. Repeats himself too much, I gave up at 1min because in that time all I got was low res display camera blue & white LED's about 5 times. Why don't they write a script that they read/follow so they don't sound like a drongo.

Leave a Reply

Your email address will not be published. Required fields are marked *