Coding time<\/h2>\n
We will use a webcam as the video input to our pose estimation model and show the output on our main page\u00a0<\/span> We are using two libraries here:<\/p>\n I\u2019ve added extensive documentation inside the code, explaining every single line. Here, we will discuss the main crux which is the majority of the code anyway.<\/p>\n Our code consists of two files:<\/p>\n The moment pose is detected and output is ready. It calls\u00a0<\/span> We store this in our\u00a0<\/span> All we have left to do is to show the image with all the detection results stored in\u00a0<\/span> As we know, the\u00a0<\/span> It takes five arguments:<\/p>\n We then call\u00a0<\/span> As you can see above, PoseNet returns a JavaScript object as output, consisting of many key-value pairs. This is the\u00a0<\/span> We have a function to draw detected points on the image. Remember, we saved all the results from the PoseNet output in the\u00a0<\/span> We loop through every\u00a0<\/span> We only draw a\u00a0<\/span> Then, we call\u00a0<\/span> Similarly, as our variable\u00a0 This is the main page where we display our output. We add all our libraries using script tags.<\/p>\nindex.html<\/code>.<\/p>\n\n
ml5.js<\/code>\u00a0<\/span>for creating and running our ML model.<\/li>\np5.js<\/code>\u00a0<\/span>for getting the webcam video feed and displaying output in our browser.<\/li>\n<\/ul>\n\n
poseNet_webcam.js<\/code>, our JavaScript code.<\/li>\nindex.html<\/code>, the main page to show output.<\/li>\n<\/ul>\nPoseNet_webcam.js<\/h2>\n
p5.js<\/code>\u00a0runs two functions:<\/p>\n\n
function setup()<\/code>. The first function that is executed and runs only once. We will do our initial setup in it.<\/li>\nfunction draw()<\/code>. This function is called on repeat forever (unless you plan on closing the browser or pressing the power button).<\/li>\n<\/ul>\n![](\"https:\/\/miro.medium.com\/max\/555\/1*7XPisaXgva97r2beNrfrpg.png\")
<\/div>\n<\/div>\n<\/div>\n<\/figure>\ncreateCanvas(width, height)<\/code>\u00a0<\/span>is provided by p5 to create a box in the browser to show our output. Here, canvas has\u00a0<\/span>width: 640px<\/code>\u00a0<\/span>and\u00a0<\/span>height: 480px<\/code>.<\/p>\ncreateCapture(VIDEO)<\/code>\u00a0<\/span>is used to capture a webcam feed and return a p5 element object, which we will name\u00a0<\/span>webcam_output<\/code>. We set the webcam video to the same height and width of our canvas.<\/p>\nml5.poseNet()<\/code>\u00a0<\/span>creates a new PoseNet model, taking as input:<\/p>\n\n
index.html<\/code>\u00a0<\/span>file, we have created an HTML paragraph with an ID\u00a0status<\/code>\u00a0<\/span>showing the current status text to the user. We change that text to\u00a0<\/span>Model Loaded\u00a0<\/span><\/em>for the user to know, as the model takes a bit to load.<\/em>\n<\/li>\n<\/ul>\nposeNet.on()<\/code>\u00a0<\/span>is a trigger or event listener. Whenever the webcam gives a new image, it is given to the PoseNet model.<\/p>\nfunction(results)<\/code>, where\u00a0<\/span>results<\/code>\u00a0is the final output of\u00a0<\/span>keypoints<\/code>\u00a0<\/span>and scores given by the model.<\/p>\nposes<\/code>\u00a0<\/span>array, which is globally defined and can be used anywhere in our code.\u00a0<\/span>webcam_output.hide()<\/code>\u00a0<\/span>hides the webcam output for now, as we will modify the images and show the image with detected points and lines later.<\/p>\n![](\"https:\/\/miro.medium.com\/max\/490\/1*aepbsCeUatjTvS2zSaA06Q.png\")
<\/div>\n<\/div>\n<\/div>\n<\/figure>\nposes<\/code>\u00a0<\/span>in the browser.<\/p>\ndraw()<\/code>\u00a0<\/span>function runs in a loop forever. Inside this, we call the\u00a0<\/span>image()<\/code>\u00a0<\/span>function to display our image (as we have our video image-by-image) in the canvas.<\/p>\n\n
input image<\/code>. The image we want to display.<\/li>\nx position<\/code>. The x-coordinate of the top-left corner of the image in respect to the canvas.<\/li>\ny position<\/code>. The y-coordinate of the top-left corner of the image in respect to the canvas.<\/li>\nwidth<\/code>. The width to draw the image.<\/li>\nheight<\/code>. The height to draw the image.<\/li>\n<\/ul>\ndrawKeyPoints()<\/code>\u00a0<\/span>and\u00a0<\/span>drawSkeleton()<\/code>\u00a0<\/span>to draw the\u00a0<\/span>dots and lines\u00a0<\/span><\/em>on the current image.\u00a0<\/span>draw()<\/code>\u00a0<\/span>does this in an infinite loop, hence showing a continuous output to the user, which makes it look like a video.<\/p>\n![\"\"](\"https:\/\/miro.medium.com\/max\/809\/1*dClYxLdf-7b-TK4QigPdBg.png\")
pose<\/code>\u00a0<\/span>key-value out of the\u00a0<\/span>pose<\/code>\u00a0<\/span>and\u00a0<\/span>skeleton<\/code>\u00a0<\/span>values, provided for each person in an image.<\/p>\n![](\"https:\/\/miro.medium.com\/max\/650\/1*dMS9-tknztdcSumiP59zGQ.png\")
<\/div>\n<\/div>\n<\/div>\n<\/figure>\nposes<\/code>\u00a0<\/span>array. Here, we loop through every\u00a0<\/span>pose<\/code>\u00a0<\/span>or person in an image and get its\u00a0<\/span>keypoints<\/code>.<\/p>\n![\"\"](\"https:\/\/miro.medium.com\/max\/501\/1*4_Hv2UqmXKE9sNZME0TSlg.png\")
point<\/code>\u00a0<\/span>that is a body part in the\u00a0<\/span>keypoints<\/code>\u00a0<\/span>array, which further has:<\/p>\n\n
part<\/code>. The name of the part detected.<\/li>\nposition<\/code>. x and y values of a point in the image.<\/li>\nscore<\/code>. Accuracy of detection.<\/li>\n<\/ul>\npoint<\/code>\u00a0if the accuracy of detection is greater than\u00a00.2<\/code>. We call\u00a0<\/span>fill(red, green, blue)<\/code>, taking RGB intensity value ranging from\u00a00 <\/code><\/span>to 255<\/code>\u00a0<\/span>to decide the color of a point, and\u00a0<\/span>noStroke()<\/code>\u00a0<\/span>to disable drawing the outline that p5 draws by default.<\/p>\nellipse(x_value, y_value, width, height)<\/code>\u00a0<\/span>to draw an ellipse at the desired position but we keep the width and height very small, which makes them look like a dot (exactly what we wanted).<\/p><\/p>\nposes<\/code>\u00a0<\/span>has multiple\u00a0pose<\/code>\u2018s in it, it also has multiple\u00a0skeleton<\/code>\u00a0values with their own type of key-value pairs, which is handled by\u00a0<\/span>drawSkeleton()<\/code>\u00a0<\/span>drawing lines instead of points.<\/p>\nindex.html<\/h2>\n
![\"\"](\"https:\/\/miro.medium.com\/max\/953\/1*zO7VSOAv82soG2gYu0l5Qw.png\")
![](\"https:\/\/miro.medium.com\/max\/493\/1*mMQwuHWBCmqKm2bbh_vlhQ.png\")
<\/div>\n<\/div>\n<\/div>\n<\/figure>\n![](\"https:\/\/miro.medium.com\/max\/600\/1*pVPUKC-SPT2Vg4vLWdN6iA.gif\")
<\/div>\n<\/div>\n<\/div>\n<\/figure>\n![\"\"](\"https:\/\/miro.medium.com\/max\/600\/1*RPBw9EuFTWPtVWcGdHYG-Q.gif\")