Perceptron Perceiving

Source: Deep Learning on Medium

Winter! Chill breeze with warm blankets, hot pho with rustic Merlot, and. Holiday season! For a change, I decided to spend my holidays back in the U S of A this year. Everybody tripping out, I had a lot of time to spend with myself. My kind of vacation indeed. But for the two Christmas holidays, I had all the time for myself. And the to-do checker I am, I had a ton of things I wanted to do during the week-long break. Writing, music practice, studying, side projects, books, cooking, cricket, fitness and so on. I made an elaborate plan to check most of these items off starting 23rd. The first two days went fine, as per my plan, but Christmas made my holidays even merrier. I did not get any work done and was often left contemplating my interactions with two high school kids I met at my aunt’s place in LA.

Rum-walnut-Nutella-cheese-cake! I like every word in that sentence. While I was completely focused on hogging away all the cake at my aunt’s place, “Oh, the solution to that Object-Oriented Programming exam question was to combine 3 perceptron systems and perform a logical OR from each of their outputs”, explained senior high school Ellie to her classmates Doug and my cousin Palak. My eyes lit up, jaws dropped on hearing high school kids speak about perceptrons. Yes, I was breathing in the AI-ML hype every single day in the tech industry, but I had not given thought about the depths it had reached — touching young high school children. My high school computer assignments only involved basic C programs to either perform arithmetic operations or display fancy patterns of asterisks. But that was 12 years old. I realized how outdated the education I went through then has become and thanked everyone who pushed me into the paths of constant learning to pursue higher education. I carried my cake over to the kids’ table to have a conversation about their assignments during their holidays. I instantly got the “you are so old and boring” look from Palak but I chose to get past the moment. “So Ellie, what did you build in the perceptron experiment?”, I started.
Over the next one hour, young Ellie would make me realize, it was not just what I had learned that was archaic, more importantly, it was the process and protocols used in schools around me which are too bizarre and counter-intuitive from a modern lens. Almost irrelevant. And the sad truth, it is over a decade and my cousins in India, are still building Pascal’s triangle of asterisks in their 101 programming assignments.

“A perceptron is a mathematical representation of a neuron. It fires when the strength of the input signal is higher than a particular threshold. An easier way to imagine can be — a perceptron is like a function that represents a line. Things to the right of a line represent crossing the threshold and those to the left represent weak input signals not strong enough to cross the threshold. The line acts as a separator. And since any line can be represented with its slope and intercept, a perceptron can also be represented with a slope and intercept. For any given input coordinates, the perceptron function tells us if the coordinate is to the left or right of the line i.e. in terms of neurons whether it crosses the threshold”, the kids enunciated.

Source Illustration of a line behaving as a separator for all points in the XY plane. The orientation of line can be configured using its slope and intercept and hence any linear separator can be represented as a line.

Sure it was mostly textbook knowledge, what most of you might already know. But what stood out was how students were asked to apply concepts of coordinate geometry learned earlier in high school, to code a bunch of plug-gable functions as part of their current programming course to build a mathematical representation of a physical system and eventually understand fundamentals of a future concept — classifiers.

Ellie went on to show me how her system of perceptrons could be extended to represent any region in the XY plane. Creating multiple perceptron functions (lines), each with different slopes and intercepts we can create closed convex polygons. Programmatically, for any input (x, y) if the point lies on a certain side of EACH of the created perceptron line, we can determine if the point is inside or outside the polygon. I was only in awe by their level of understanding and application.

Palak’s calculations — The hexagon is identified by its 6 bounding lines. For each line the equation was derived using points of intersection. Each line was modeled as a separator to represent the region inside the hexagon. The conditions are show in green and blue.