I have an arducam mini 2MP camera connected to an ESP8266 (12-E) module and I am trying to implement video streaming inside a window with some text and control buttons around it, all in the same browser tab /page. I have created two HTML pages for the server to use. The first is the home webpage with no image streaming, just a simple page with text buttons and some CSS. The second HTML page serves the continuous frames (streaming video) along with some text and buttons to the browser. When the home page is sent to the browser everything is displayed the way I expect. But, when the second HTML webpage is served but some strange things happen when the browser(Firefox or Chrome) receives the reply from the server (esp12-e).
Normally I would expect a small window displaying continuous frames taken from the camera with some text over that window an some control buttons under it. But, instead of that two things happen.
- Only the video streaming window is displayed in the browser's tab but around this window there is only a gray background color. No buttons no text. When I open the HTML Inspector, inside "head", there are a few lines of HTML code that create the background gray color and some CSS stuff I have not written in my server. Somehow the browser creates this lines of code automatically and adds them in my original HTML code.
- In my original HTML code, inside "body", along with the code for the streaming window, I have the code for the text and button elements that will be displayed. But in the browser, these parts disappear. When I open the Inspector these elements don't exist! I have tried various approaches so far to avoid this situation by isolating/embedding the streaming window inside the browser's tab. These approaches are: iframe, data URI, multipart/x-mixed replace, form. Unfortunately the same result occured for all of these approaches ( gray background color, streaming window in the middle of the screen and disappeared buttons and text ).
The only thing I know is that, when the browser "sees" the incoming image from the server it produces these side effects. When I create an HTML only with text and buttons it is displayed just fine. I do something the wrong here but I cannot find what it is.
Below I attach 2 pictures of what I get in the browser's tab and the HTML code I send from the esp-12e server for a photo capture
void serveWebpage(WiFiClient client){
String answer = "HTTP/1.1 200 OK\r\n";
answer += "Content-Type: text/html\r\n\r\n";
answer +="<!DOCTYPE HTML>\r\n";
answer += "<html>\r\n";
answer +="<head><title> Monitor </title></head>\r\n";
answer += "<body>\r\n";
answer += "<h1 style=\"position:relative; left:25px;\"> ⚓ Observation Panel ⚓</h1>\r\n"; // Header Text
answer += "<a href=\"/videoStream\"><button type=\"button\" style=\"position:absolute; top:340px;"; // First Button
answer += "left:95px; color:blue; height:70px; width:90px; font-weight: bold; border-style:outset;";
answer += "border-width:2px; border-color:black;\"> Video Stream </button></a>\r\n";
answer += "<a href=\"PhotoCapture\"><button type=\"button\" style=\"position:absolute; top:340px;"; // Second Button
answer += "left:195px; color:blue; height:70px; width:90px; font-weight: bold; border-style:outset;";
answer += "border-width:2px; border-color:black;\"> Video Stream </button></a>\r\n";
answer += "<div>\r\n";
answer += "<img src='data:image/jpeg; charset=utf-8; base64,"; // Here the image is wrapped with data URI to display it in the browser
myCAM.clear_fifo_flag(); // this part is taken from the arducam library exammples. It captures the image and sends it to browser
myCAM.start_capture();
while (!myCAM.get_bit(ARDUCHIP_TRIG, CAP_DONE_MASK)); // wait here until capture has completed
size_t len = myCAM.read_fifo_length();
myCAM.CS_LOW();
myCAM.set_fifo_burst();
#if !(defined (ARDUCAM_SHIELD_V2) && defined (OV2640_CAM))
SPI.transfer(0xFF);
#endif
static const size_t bufferSize = 4096; //4096
static uint8_t buffer[bufferSize] = {0xFF};
while (len) {
size_t will_copy = (len < bufferSize) ? len : bufferSize;
SPI.transferBytes(&buffer[0], &buffer[0], will_copy);
if (!client.connected()) break;
client.write(&buffer[0], will_copy);
len -= will_copy;
}
myCAM.CS_HIGH();
answer +="9k=' />"; // closing the <img>
answer +="</div>\r\n";
answer +="</body>\r\n";
answer +="</html>\r\n\r\n";
client.print(answer);
}
I made some progress finally but not 100%. I managed to display jpeg images in the iframe by embedding data in jpeg format from an image with data URI method inside Iframe element.
string = "<iframe srcdoc='<img src=\"data:html/text;base64,/9j/4AAQ..... \" > ' > ";
My mistake was that I did not use the quotes with the correct order and image data were interpreted as text in the browser. Then I tried to do the same thing with the function I use to send the captured image from the camera to the browser. Unfortunately the same issue arose and I cannot fix it this time. Something happens when I send a string with multiple quotes to the browser because it interprets them as text and not as jpeg data format like this: /9j/4AAQ...... I uploaded a picture from my browser's inspector(showing the browser's received data when I use the function for camera's sent frame data) to figure out easier what I mean. Any ideas on this ?
Here is a review of what I have completed so far. I created an HTML with an Iframe inside it and also some buttons. Both iframe and buttons are displayed correctly in the same browser's tab. Now, Inside iframe I put srcdoc attribute and inserted the raw jpeg data directly in there (of a sample jpeg image) since they are encoded (base64) but the browser interpreted these jpeg data as plain text and displayed them in the iframe as text. Then I used the image tag inside srcdoc to wrap the raw jpeg data in the iframe. This worked after some mistakes I did with the quotes inside iframe string. Then I removed the raw jpeg data from the image tag and replaced them with the function that brings jpeg data from the camera. I send the first part of the answer string (opening iframe and img tags), then I send the data from the camera and finally I send the second part of the answer string (closing iframe and img tags). Normally it should work since I followed the same procedure as before. But the browser could not interpret the image ...again.
Below, I have added the code parts for the encoded sample image (that the browser interpreted as image) and then the camera function (which interpreted them as odd characters and not image), for comparison. Both should work the same way but only the first works.
Encoded sample image:
answer = "<iframe srcdoc='<img src=\"data:image/jpeg;base64,/9j/4AAQS...0KDQo=\"> ' scrolling=\"no\" width=\"340\" height=\"340\" > <p> Error </p> </iframe>\r\n ";
Camera function sendFrame():
answer = "<iframe srcdoc=\"<img src='data:image/jpeg;base64,";
client.print(answer);
sendFrame();
answer ="' > \" > <p> Error </p> </iframe>\r\n ";
client.print(answer);
So, I think I have found what goes wrong with the camera's incoming jpeg data The camera function brings the jpeg data (to the server and then to the client) in a format that the browser interprets it as text or something like that because it contains strange characters (check the last image I posted).
Also in order to write the html code I use quotes " and ' (or " and \') to create the iframe code and everything else inside iframe.
And here is the thing: Because some of the camera's jpeg data are interpreted as quotes by the browser, they interact with the quotes I put inside iframe to wrap the img tag and the data coming from the camera and That is why it messes everything up in iframe (I think)
Is there anyway to convert the image data coming from the camera function into base64, so as they don't interact with the wrapping quotes of the iframe and image tags?
Content-Type, as it's clearly being interpreted as an image (all the inserted styles are internal Firefox styles)