Artificial Intelligence has become part of our everyday lives, and many AI features send your data to the internet for processing. That’s where on device AI, also known as Edge AI, comes in. Instead of relying on cloud servers, the AI runs directly on your device, making phones, computers, watches, TVs, cameras, and even coffee makers and air conditioners smarter while keeping your data local.
In this article, we will learn what a machine learning (ML) model is, how it works, and how to use one in your application. You can think of an ML model as a processing unit with an input, a process, and an output. We’ll explore what the input looks like, how to feed data into a model, and how to understand its output. You’ll also see that both input and output are simply collections of numbers arranged in multidimensional arrays (called Tensors).
What’s this article not about**?**
Machine Learning vs Deep Learning
All machine learning models are programs that have learned how to perform a specific task by studying a large amount of data. Unlike a normal program, where we write all the rules, an ML model learns those rules during training.
Once the training is finished, they become static; from that point on, it only performs predictions from new input data, also known as **Inference **or Interpreting.
The following hierarchy explains the relationship among AI, ML, and DL:
Artificial Intelligence (AI) Rules and Logic // Can not be trained during usage (static)└── Machine Learning (ML) replaces rules with experience // Can be trained during usage (dynamic)└── Deep Learning (DL) automatic learning
Every machine learning model follows the same pipeline.
Input ↓Process ↓Output
The input is the data you provide to the model.
Depending on the model, the input could be:
However, a model cannot understand any of these directly. Before the data reaches the model, it must be converted into numbers called Tensor.
A tensor is simply a collection of numbers.
If you’ve worked with arrays in programming, you’ve already used something very similar. The only difference is that a tensor can have more than one dimension. It’s simply a matrix.
1D Tensor[12, 34, 56]2D Tensor[ [1, 2, 3], [4, 5, 6]]
What Does Shape Mean?
The shape of a tensor describes how its numbers are organized.
It tells us the size of each dimension, but it does not change the actual values stored inside the tensor.
For example, a 1D tensor:
[12, 34, 56] -> shape is [3]
A 2D tensor:
[ [1, 2, 3], [4, 5, 6]]This tensor has two dimensions:2 rows3 columnsShape is [2,3]
What if the model receives an image?
Example for Face Detection Input
Suppose we have a face detection model that expects an RGB image with a size of 320 × 320 pixels.
The original image:
320 × 320 RGB Image
will be converted into a tensor with the shape of:
[1, 320, 320, 3]
where:
But internally, this is just one long array containing:
1 x 320 × 320 × 3 = 307,200 numbers
Check out this code example on converting a Bitmap to a Tensor.
But what if the model receives text?
Example for Text Model Input
Not every model works with images. Some models work with text instead.
For example, an English Grapheme-to-Phoneme** (G2P)** model converts written words into their pronunciation sounds.
A model like DeepPhonemizer receives text as input and predicts the phonemes (the basic sound units of a language).
The input might be:
"hello"
When we convert this to a Tensor related to the model input shape, it will become something like this: (each char will become a number)
[ [ 21, 8, 15, 15, 18 ]]
The model receives this data as a tensor with shape of:
text [1, 96]
This means:
If the text is shorter than 96 tokens, the remaining positions are usually filled with padding values.
The process is the work performed inside the machine learning model.
Here, the model applies the knowledge it learned during training by performing thousands or even millions of mathematical operations on the input tensor.
The output is the model’s prediction.
Depending on the model, the output might be:
Just like the input, the output is also a tensor with a predefined shape.
Example: Face Detection Output
Suppose the model detects two faces in the image.
Instead of returning rectangles or drawing anything on the screen, it returns a tensor containing numbers such as:
[ 45, 62, 128, 180, 0.98, 185, 74, 276, 192, 0.95]
For a real face detection model, the output tensor might have the shape:
[1, 2500, 5]
This means:
You can visualize the output tensor like this:
[ [ ← Batch (1 image) [45, 62, 128, 180, 0.98], ← Prediction 1 [185, 74, 276, 192, 0.95], ← Prediction 2 [ ... ], ← Prediction 3 ... [ ... ] ← Prediction 2500 ]]
in code, this will look like:
val output = Array(1) { Array(2500) { FloatArray(5) }}
so the shape of our model output is [1,2500,5].
you can access the first prediction like this:
val firstPrediction = output[0][0]val x = firstPrediction[0]val y = firstPrediction[1]val width = firstPrediction[2]val height = firstPrediction[3]val confidence = firstPrediction[4]
Most of these 2,500 predictions are not actual faces. Many will have very low confidence scores, such as 0.02 or 0.15. You must remove low confidence predictions and keep only the best ones. This is also known as Thresholding.
But what if the model produces text?
Example for Text Model Output
Consider the input we earlier gave in the input section. The model then processes that input tensor and produces an output tensor with a shape of:
[1, 96, 64]
This means:
Here, the output is called logits. Logits are raw prediction scores before they are converted into probabilities.
For example, for the first character position, the model might output:
[ 0.2, -1.5, 3.8, ...]
After converting these predictions, the output might become:
hello ↓ G2PHH AH L OW
Normalization is the process of changing numerical values into a specific range or format that the machine learning model expects. Almost 90% of errors will hover around this section.
Normalization can happen in two places:
Now that we understand how data moves through a machine learning model, the next step is running these models inside an application.
For edge AI, models are usually converted into formats optimized for mobile and embedded devices, such as tflite, onnx, or pt.
One of the most common libraries is TensorFlow Lite, now evolving under the name LiteRT.
Another popular option is PyTorch Runtime or ExecuTorch:
Thank you for reading.
Learn about ‘Edge AI’ like a child was originally published in Dev Genius on Medium, where people are continuing the conversation by highlighting and responding to this story.