From mathematical foundations to edge implementation
👨🏽💻 Github: thommaskevin/TinyML (github.com)
👷🏾 Linkedin: Thommas Kevin | LinkedIn
📽 Youtube: Thommas Kevin — YouTube
👨🏻🏫 Analysis group: Conecta.ai (ufrn.br)
SUMMARY
1 — Poisson Regression Concept
1.1 — Assumptions
1.2 — The Mannequin
1.3 — Estimating Coefficients
2 — TinyML Implementation
Poisson regression fashions are utilized to depict occasions the place outcomes are counts: discrete knowledge with non-negative integer values that tally one thing, such because the variety of instances an occasion happens throughout a sure time frame or the variety of folks within the grocery store queue.
Rely knowledge can be expressed as fee knowledge, because the variety of instances an occasion happens inside a time interval will be expressed as a uncooked rely, as an illustration, we eat three meals a day.
Poisson regression aids us in analyzing rely knowledge and fee knowledge, enabling us to determine which explanatory variables affect a selected response variable Y, whether or not it’s a rely or a fee. As an example, Poisson regression is perhaps utilized by a grocery store to higher perceive and predict the variety of folks in a queue.
The Poisson distribution fashions the likelihood of occasion or occasions Y occurring inside a particular time interval, assuming that the occurrences Y are usually not influenced by the timing of earlier occurrences of Y. This may be mathematically expressed utilizing the next expression:
the place y = 0,1,2,⋯.
Right here, μ is the common variety of instances an occasion can happen per unit of publicity. Additionally it is known as the Poisson distribution parameter. Publicity will be time, house, inhabitants measurement, distance, or space, however it’s sometimes time, denoted as (t). If the publicity worth shouldn’t be supplied, it will likely be assumed to be equal to 1.
1.1 — Assumptions
Poisson Regression, like all statistical mannequin, comes with sure assumptions which might be necessary to think about when utilizing the mannequin and deciphering its outcomes. Listed below are the important thing assumptions of Poisson Regression:
A. Independence of Observations: The observations needs to be unbiased of one another. Which means that the incidence of an occasion in a single statement mustn’t affect the incidence of an occasion in one other statement. If there’s dependence amongst observations, it could result in biased parameter estimates and incorrect inferences.
B. Linearity in Parameters: The connection between the unbiased variables and the log-transformed imply of the Poisson distribution needs to be linear. This assumption implies that the impact of every unbiased variable on the dependent variable is fixed throughout completely different values of the unbiased variables.
C. Absence of Overdispersion: Poisson Regression assumes that the variance of the dependent variable is the same as its imply. Nonetheless, in real-world situations, it’s frequent to come across conditions the place the variance exceeds the imply, a situation generally known as overdispersion. If overdispersion is current, it could result in inefficient parameter estimates and underestimated commonplace errors.
D. Right Specification of Mannequin: It’s essential to incorporate all related unbiased variables within the mannequin and to appropriately specify the practical type of the mannequin. Omitting necessary variables or utilizing an incorrect practical kind can result in biased parameter estimates and inaccurate predictions.
E. Rely Knowledge: Poisson Regression is appropriate for modeling rely knowledge, the place the dependent variable represents the variety of occurrences of an occasion inside a hard and fast unit of time or house. Utilizing Poisson Regression for non-count knowledge might violate the underlying assumptions of the mannequin and result in unreliable outcomes.
F. No Multicollinearity: There needs to be no multicollinearity among the many unbiased variables. Multicollinearity happens when two or extra unbiased variables are extremely correlated, making it troublesome to estimate their particular person results on the dependent variable. Excessive multicollinearity can inflate commonplace errors and result in unstable parameter estimates.
G. No Outliers: Outliers within the knowledge can unduly affect the parameter estimates and have an effect on the general match of the mannequin. It’s necessary to determine and appropriately deal with outliers to make sure the validity of the regression outcomes.
1.2— The Mannequin
Generalized Linear Fashions (GLMs) are fashions through which the response variables observe a distribution aside from the traditional distribution. This contrasts with linear regression fashions, the place the response variables observe a traditional distribution. It’s because Generalized Linear Fashions have response variables which might be categorical, reminiscent of Sure, No; or Group A, Group B, and thus don’t vary from −∞ to +∞.
Due to this fact, the connection between the response and predictor variables might not be linear. In generalized linear fashions:
the place g(⋅) is the chosen hyperlink operate.
A Poisson regression mannequin is a generalized linear mannequin used to mannequin rely knowledge and contingency tables. The output Y (rely) is a price that follows the Poisson distribution. It assumes the logarithm of the anticipated values (imply) which will be modeled in a linear kind by some unknown parameters.
To rework the non-linear relationship into linear kind, a hyperlink operate is used, which is the logarithm of the Poisson regression. For that reason, a Poisson regression mannequin can be known as a log-linear mannequin. The overall mathematical type of the Poisson regression mannequin is:
the place μi are the expectations of the response variables Yi.
The coefficients α and β are numerical, the place α is the intercept, typically α can be represented by β0, and x are the predictor or explanatory variables.
Take into account an equation with one predictor variable and one response variable:
That is equal to:
In Poisson regression fashions, predictor or explanatory variables can have a combination of numerical or categorical values.
One of the vital necessary options of the Poisson distribution and Poisson regression is equidispersion, which signifies that the imply and variance of the distribution are equal.
Let’s say the imply E(Y) = μ. For Poisson regression, the imply and variance are associated as follows:
the place σ² is the dispersion parameter. Since var(Y) = E(Y) (variance = imply) should maintain for the Poisson mannequin to be absolutely fitted, σ² ought to equal 1.
When the variance is bigger than the imply, that is known as overdispersion and σ² is bigger than 1. If σ² is lower than 1, then it is called underdispersion.
1.3 — Estimating Coefficients
In Poisson Regression, the mannequin parameters are estimated to search out the most effective relationship between the unbiased variables and the dependent rely variable. This estimation is often completed utilizing the strategy of most probability.
1.3.1 — Most Chance Technique (MLM)
The strategy of most chances are a broadly used statistical method for estimating the parameters of a statistical mannequin. It seeks to search out the parameter values that maximize the probability of the noticed knowledge, i.e., we search the parameters that make the noticed knowledge almost certainly to have been generated by the proposed mannequin.
In Poisson Regression, the probability operate is outlined because the product of the chances of observing the noticed counts for every particular person statement. Formally, the probability operate L(β0, β1,…, βp) is given by:
The place:
- n is the overall variety of observations;
- yi is the noticed rely for the i-th statement;
- μi is the theoretical imply akin to the i-th statement, given by the logarithmic hyperlink operate.
The target of parameter estimation is to search out the values of the coefficients β0, β1,…, βp that maximize the probability operate. In different phrases, we search the parameters that make the noticed knowledge almost certainly to have been generated by the Poisson Regression mannequin.
With this instance you’ll be able to implement the machine studying algorithm in ESP32, Arduino, Arduino Portenta H7 with Imaginative and prescient Protect, Raspberry and different completely different microcontrollers or IoT gadgets.
2.0 — Set up the libraries listed within the necessities.txt file
!pip set up -r necessities.txt
2.1 — Importing libraries
from sklearn.model_selection import train_test_split
from sklearn.linear_model import PoissonRegressor
from sklearn.metrics import (
mean_absolute_error,
mean_poisson_deviance,
mean_squared_error,
)import m2cgen as m2c
import numpy as np
import pandas as pd
import seaborn as sns
from matplotlib import pyplot as plt
import warnings
warnings.filterwarnings('ignore')
2.2 — Load Dataset
The “Automobile Attributes and Emissions Dataset” incorporates complete info on varied automobiles manufactured within the 12 months 2000. It consists of particulars reminiscent of make, mannequin, automobile class, engine measurement, cylinder rely, transmission kind, and gasoline kind. Moreover, the dataset supplies ranges for gasoline consumption and CO2 emissions, providing insights into the environmental impression of every automobile. The dataset encompasses a variety of auto varieties, from compact to mid-size, and consists of each standard and high-performance fashions. With this info, analysts and researchers can research developments in automobile traits, gasoline effectivity, and emissions . This dataset serves as a worthwhile useful resource for understanding the automotive panorama and informing discussions on environmental sustainability and transportation insurance policies.
hyperlink: https://www.kaggle.com/datasets/krupadharamshi/fuelconsumption/data
df = pd.read_csv('./knowledge/FuelConsumption.csv')
df.head()
df.information()
df.describe()
2.3 —Clear Knowledge
# 1. Eradicating rows with lacking values
df.dropna(inplace=True)
# 2. Eradicating duplicates if any
df.drop_duplicates(inplace=True)
# Show the dataframe after cleansing
df.describe()
2.4 — Exploratory Knowledge Evaluation
sns.pairplot(df[['ENGINE SIZE','CYLINDERS','FUEL CONSUMPTION','COEMISSIONS ']])
plt.savefig('.figurespairplot.png', dpi=300, bbox_inches='tight')
corr = df[['ENGINE SIZE','CYLINDERS','FUEL CONSUMPTION','COEMISSIONS ']].corr('spearman')
# Adjusting the dimensions of the determine
plt.determine(figsize=(18,10))
# Your current code for producing the heatmap
heatmap = sns.heatmap(corr, xticklabels=corr.columns, yticklabels=corr.columns, cmap='coolwarm')
# Including values to the heatmap
for i in vary(len(corr.columns)):
for j in vary(len(corr.columns)):
plt.textual content(j + 0.5, i + 0.5, f"{corr.iloc[i, j]:.2f}", ha='middle', va='middle', shade='black', fontsize=18)plt.xticks(fontsize=20, rotation=45)
plt.yticks(fontsize=20, rotation=0)
cbar = heatmap.collections[0].colorbar
cbar.ax.tick_params(labelsize=20)
plt.savefig('.figuresheatmap.png', dpi=300, bbox_inches='tight')
# Show the heatmap
plt.present()
2.5— Cut up into coaching and take a look at knowledge
X=df[['ENGINE SIZE','CYLINDERS', 'COEMISSIONS ']]
y=df[['FUEL CONSUMPTION']]
# Cut up the information into coaching and take a look at units
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=42)
2.6 — Create the regressor mannequin
def score_estimator(y_pred , y_true):print(
"MSE: %.3f"
% mean_squared_error(
y_true, y_pred
)
)
print(
"MAE: %.3f"
% mean_absolute_error(
y_true, y_pred,
)
)
# Ignore non-positive predictions, as they're invalid for
# the Poisson deviance.
masks = y_pred > 0
if (~masks).any():
n_masked, n_samples = (~masks).sum(), masks.form[0]
print(
"WARNING: Estimator yields invalid, non-positive predictions "
f" for {n_masked} samples out of {n_samples}. These predictions "
"are ignored when computing the Poisson deviance."
)
print(
"imply Poisson deviance: %.3f"
% mean_poisson_deviance(
y_true ,
y_pred
)
)
mannequin = PoissonRegressor(alpha=1e-12)
2.7 —Prepare the mannequin
mannequin.match(X_train, y_train)
2.8 — Mannequin analysis
y_train_pred = mannequin.predict(X_train)
y_test_pred = mannequin.predict(X_test)
# Calculate residuals
train_residuals = y_train.values.reshape(1,-1).tolist()[0] - y_train_pred
# Calculate imply and commonplace deviation of residuals
train_residuals_mean = np.imply(train_residuals)
train_residuals_std = np.std(train_residuals)
# Calculate residuals
test_residuals = y_test.values.reshape(1,-1).tolist()[0] - y_test_pred
# Calculate imply and commonplace deviation of residuals
test_residuals_mean = np.imply(test_residuals)
test_residuals_std = np.std(test_residuals)# Plot residuals
plt.determine(figsize=(10, 5))
plt.subplot(1, 2, 1)
plt.scatter(y_train_pred, train_residuals, c='blue', marker='o', label=f'Coaching knowledge')
plt.axhline(y=0, shade='r', linestyle='-')
plt.axhline(y=train_residuals_mean, shade='ok', linestyle='--', label=f'Imply: {train_residuals_mean:.3f}')
plt.axhline(y=train_residuals_mean + 2 * train_residuals_std, shade='g', linestyle='--', label=f'+2 Std Dev: {2*train_residuals_std:.2f}')
plt.axhline(y=train_residuals_mean - 2 * train_residuals_std, shade='g', linestyle='--', label=f'-2 Std Dev: {-2*train_residuals_std:.2f}')
plt.xlabel('Predicted values')
plt.ylabel('Residuals')
plt.title('Residuals vs Predicted values (Coaching knowledge)')
plt.legend(loc='higher left')
plt.grid(True)
plt.subplot(1, 2, 2)
plt.scatter(y_test_pred, test_residuals, c='inexperienced', marker='s', label=f'Check knowledge')
plt.axhline(y=0, shade='r', linestyle='-')
plt.axhline(y=test_residuals_mean, shade='ok', linestyle='--', label=f'Imply: {test_residuals_mean:.3f}')
plt.axhline(y=test_residuals_mean + 2 * test_residuals_std, shade='g', linestyle='--', label=f'+2 Std Dev: {2*test_residuals_std:.2f}')
plt.axhline(y=test_residuals_mean - 2 * test_residuals_std, shade='g', linestyle='--', label=f'-2 Std Dev: {-2*test_residuals_std:.2f}')
plt.xlabel('Predicted values')
plt.ylabel('Residuals')
plt.title('Residuals vs Predicted values (Check knowledge)')
plt.legend(loc='higher left')
plt.grid(True)
plt.tight_layout()
plt.present()
# Verify for normality
plt.determine(figsize=(10, 5))
plt.subplot(1, 2, 1)
plt.hist(train_residuals, bins=20, shade='blue', alpha=0.6)
plt.title('Histogram of Residuals (Coaching knowledge)')
plt.xlabel('Residuals')
plt.ylabel('Frequency')
plt.axvline(x=train_residuals_mean, shade='ok', linestyle='--', label=f'Imply: {train_residuals_mean:.3f}')
plt.axvline(x=train_residuals_mean + 2 * train_residuals_std, shade='g', linestyle='--', label=f'+2 Std Dev: {2*train_residuals_std:.3f}')
plt.axvline(x=train_residuals_mean - 2 * train_residuals_std, shade='g', linestyle='--', label=f'-2 Std Dev: {-2*train_residuals_std:.3f}')
plt.legend(loc='higher proper')
plt.grid(True)
plt.subplot(1, 2, 2)
plt.hist(test_residuals, bins=20, shade='inexperienced', alpha=0.6)
plt.title('Histogram of Residuals (Check knowledge)')
plt.xlabel('Residuals')
plt.ylabel('Frequency')
plt.axvline(x=test_residuals_mean, shade='ok', linestyle='--', label=f'Imply: {test_residuals_mean:.3f}')
plt.axvline(x=test_residuals_mean + 2 * test_residuals_std, shade='g', linestyle='--', label=f'+2 Std Dev: {2*test_residuals_std:.3f}')
plt.axvline(x=test_residuals_mean - 2 * test_residuals_std, shade='g', linestyle='--', label=f'-2 Std Dev: {-2*test_residuals_std:.3f}')
plt.legend(loc='higher proper')
plt.grid(True)
plt.tight_layout()
plt.present()
2.8.1 — Evaluating the mannequin with practice knowledge
print("PoissonRegressor analysis:")
score_estimator(y_train_pred, y_train)
plt.plot(y_train.values, label="unique")
plt.plot(y_train_pred, label="predicted")
plt.legend(loc='finest',fancybox=True, shadow=True)
plt.grid()
2.8.2 — Evaluating the mannequin with take a look at knowledge
print("PoissonRegressor analysis:")
score_estimator(y_test_pred, y_test)
plt.plot(y_test.values, label="unique")
plt.plot(y_test_pred, label="predicted")
plt.legend(loc='finest',fancybox=True, shadow=True)
plt.grid()
2.9 — Acquiring the mannequin to be carried out within the microcontroller
code = m2c.export_to_c(mannequin)
print(code)
#embody <math.h>
double rating(double *enter)
{
return exp(1.7347124654302846 + enter[0] * 0.011406244946132144 + enter[1] * 0.01010646886054758 + enter[2] * 0.0028201461971878914);
}
2.10— Saves the template in a .h file
with open('./PoissonRegressor.h', 'w') as file:
file.write(code)
2.11 — Deploy Mannequin
2.11.1 — Full Arduino Sketch
#embody "PoissonRegressor.h"Eloquent::ML::Port::PoissonRegressor PoissonRegressor;
void setup()
{
Serial.start(115200);
}
void loop()
{
float X_1[] = {6., 2.7, 5.1, 1.6};
int result_1 = PoissonRegressor.predict(X_1);
Serial.print("Results of predict with enter X1 (actual worth = 1):");
Serial.println(result_1);
delay(2000);
float X_2[] = {4.8, 3.1, 1.6, 0.2};
int result_2 = PoissonRegressor.predict(X_2);
Serial.print("Results of predict with enter X2 (actual worth = 0):");
Serial.println(result_2);
delay(2000);
}
2.12 — Outcomes
Full mission in: TinyML/15_Poisson_Regressor at main · thommaskevin/TinyML (github.com)
When you prefer it, think about shopping for my espresso ☕️💰 (Bitcoin)
code: bc1qzydjy4m9yhmjjrkgtrzhsgmkq79qenvcvc7qzn