Time Series & Recommendation Algorithms (9)

Forecasting future values and recommending items

Part 1: Time Series Algorithms (5)

Algorithm	Handles Seasonality?	Handles Trend?	Complexity	Best For
ARIMA	No	Yes (via differencing)	Medium	Non-seasonal stationary data
SARIMA	Yes	Yes	Medium	Seasonal time series
Prophet	Yes (multiple)	Yes (flexible)	Low	Business forecasting
Holt-Winters	Yes	Yes	Low	Smooth seasonal patterns
State Space Models	Yes	Yes	High	Complex dynamics, filtering

1. ARIMA

Description: AutoRegressive Integrated Moving Average. Combines three components: AR (autoregression on past values), I (differencing to make data stationary), and MA (moving average of past forecast errors). The model is specified as ARIMA(p, d, q) where p = AR order, d = differencing order, q = MA order.

Use Cases: Stock prices, economic indicators, demand forecasting, any univariate non-seasonal time series.

from statsmodels.tsa.arima.model import ARIMA
import pandas as pd
import numpy as np

# Generate sample time series
np.random.seed(42)
dates = pd.date_range('2020-01-01', periods=200, freq='D')
y = np.cumsum(np.random.randn(200)) + 100  # Random walk + drift
ts = pd.Series(y, index=dates)

# Fit ARIMA(2, 1, 2)
model = ARIMA(ts, order=(2, 1, 2))
results = model.fit()

# Forecast next 30 days
forecast = results.forecast(steps=30)
print(f"AIC: {results.aic:.2f}")
print(f"BIC: {results.bic:.2f}")
print(f"Forecast (next 5): {forecast.values[:5].round(2)}")

# Auto ARIMA (finds best p, d, q automatically)
# from pmdarima import auto_arima
# auto_model = auto_arima(ts, seasonal=False, stepwise=True)
# print(auto_model.summary())

2. SARIMA

Description: Seasonal ARIMA. Extends ARIMA with seasonal components, specified as SARIMA(p,d,q)(P,D,Q,s) where the uppercase letters represent seasonal AR, differencing, and MA orders, and s is the seasonal period (e.g., 12 for monthly data with yearly seasonality).

Use Cases: Monthly sales with yearly patterns, daily data with weekly patterns, any time series with clear seasonal cycles.

from statsmodels.tsa.statespace.sarimax import SARIMAX

# Generate seasonal data
np.random.seed(42)
n = 120  # 10 years of monthly data
trend = np.linspace(50, 100, n)
seasonal = 15 * np.sin(np.linspace(0, 20 * np.pi, n))  # Yearly cycle
noise = np.random.randn(n) * 3
ts_seasonal = pd.Series(
    trend + seasonal + noise,
    index=pd.date_range('2015-01-01', periods=n, freq='M')
)

# SARIMA(1,1,1)(1,1,1,12)
model = SARIMAX(
    ts_seasonal,
    order=(1, 1, 1),
    seasonal_order=(1, 1, 1, 12),
    enforce_stationarity=False,
    enforce_invertibility=False
)
results = model.fit(disp=False)

forecast = results.forecast(steps=12)
print(f"AIC: {results.aic:.2f}")
print(f"Next 3 months: {forecast.values[:3].round(2)}")

3. Prophet

Description: Developed by Facebook (Meta), Prophet is an additive regression model: y(t) = g(t) + s(t) + h(t) + error, where g(t) is trend (linear or logistic growth), s(t) is seasonality (Fourier series), and h(t) is holiday effects. Designed to be easy to use with sensible defaults.

Use Cases: Business forecasting, web traffic, retail sales, any business metric with daily/weekly/yearly patterns and holiday effects.

from prophet import Prophet
import pandas as pd

# Prophet requires columns: 'ds' (date) and 'y' (value)
df = pd.DataFrame({
    'ds': pd.date_range('2020-01-01', periods=365, freq='D'),
    'y': np.cumsum(np.random.randn(365)) + 100 + 10 * np.sin(np.linspace(0, 4*np.pi, 365))
})

model = Prophet(
    yearly_seasonality=True,
    weekly_seasonality=True,
    daily_seasonality=False,
    changepoint_prior_scale=0.05  # Flexibility of trend changes
)
model.fit(df)

# Forecast next 90 days
future = model.make_future_dataframe(periods=90)
forecast = model.predict(future)

print(forecast[['ds', 'yhat', 'yhat_lower', 'yhat_upper']].tail())
# model.plot(forecast)       # Plot forecast
# model.plot_components(forecast)  # Plot trend + seasonality

4. Holt-Winters (Triple Exponential Smoothing)

Description: Extends exponential smoothing to capture level, trend, and seasonality. Three smoothing equations update each component at every time step. Supports additive and multiplicative seasonality.

Use Cases: Short-term forecasting, retail demand, inventory planning, smooth seasonal patterns.

from statsmodels.tsa.holtwinters import ExponentialSmoothing

# Fit Holt-Winters with additive seasonality
model = ExponentialSmoothing(
    ts_seasonal,
    trend='add',              # 'add' or 'mul'
    seasonal='add',           # 'add' or 'mul'
    seasonal_periods=12       # Monthly with yearly seasonality
)
results = model.fit(optimized=True)

forecast = results.forecast(steps=12)
print(f"Smoothing parameters:")
print(f"  Alpha (level): {results.params['smoothing_level']:.4f}")
print(f"  Beta (trend): {results.params['smoothing_trend']:.4f}")
print(f"  Gamma (seasonal): {results.params['smoothing_seasonal']:.4f}")
print(f"SSE: {results.sse:.2f}")

5. State Space Models

Description: A general framework that models time series using hidden (latent) state variables that evolve over time according to a state equation, while observations are generated from the state via an observation equation. Subsumes ARIMA and exponential smoothing as special cases. Kalman filter is used for inference.

Use Cases: Navigation (GPS), economics (unobserved components), control systems, any system with latent dynamics.

from statsmodels.tsa.statespace.structural import UnobservedComponents

# Unobserved Components Model (Local Linear Trend + Seasonal)
model = UnobservedComponents(
    ts_seasonal,
    level='local linear trend',
    seasonal=12                  # Seasonal period
)
results = model.fit(disp=False)

# Extract components
print(f"Log-likelihood: {results.llf:.2f}")
print(f"AIC: {results.aic:.2f}")

# Forecast
forecast = results.forecast(steps=12)
print(f"Next 3 months: {forecast.values[:3].round(2)}")

# Kalman Filter example
# from pykalman import KalmanFilter
# kf = KalmanFilter(n_dim_obs=1, n_dim_state=2)
# state_means, state_covs = kf.em(observations).smooth(observations)

Part 2: Recommendation Algorithms (4)

Algorithm	Approach	Needs User History?	Cold Start?	Best For
Collaborative Filtering	User/item similarity	Yes	Problematic	When rich interaction data exists
Content-Based Filtering	Item features	Minimal	Handles well	New users, rich item metadata
Matrix Factorization	Latent factors	Yes	Problematic	Sparse rating matrices
Factorization Machines	Feature interactions	Flexible	Handles well	High-dimensional sparse data

6. Collaborative Filtering

Description: Recommends items based on the preferences of similar users (user-based) or similar items (item-based). The core idea: if users A and B liked the same items in the past, they will likely agree on future items too. Does not need to understand item content.

Use Cases: Movie recommendations (Netflix), product suggestions (Amazon), music playlists (Spotify).

import numpy as np
from sklearn.metrics.pairwise import cosine_similarity

# User-item rating matrix (0 = not rated)
ratings = np.array([
    [5, 3, 0, 1, 4],   # User 0
    [4, 0, 0, 1, 3],   # User 1
    [1, 1, 0, 5, 4],   # User 2
    [0, 1, 5, 4, 0],   # User 3
    [0, 0, 4, 5, 3],   # User 4
])

# User-based collaborative filtering
user_sim = cosine_similarity(ratings)
print("User similarity matrix:")
print(user_sim.round(2))

# Predict rating for User 1, Item 2
target_user, target_item = 1, 2
# Find users who rated this item
rated_mask = ratings[:, target_item] > 0
similarities = user_sim[target_user][rated_mask]
user_ratings = ratings[rated_mask, target_item]
predicted = np.dot(similarities, user_ratings) / (similarities.sum() + 1e-8)
print(f"\nPredicted rating for User {target_user}, Item {target_item}: {predicted:.2f}")

7. Content-Based Filtering

Description: Recommends items similar to those a user has liked before, based on item features (genre, description, attributes). Builds a user profile from the features of items they have interacted with, then matches new items against this profile.

Use Cases: News article recommendations, when item metadata is rich, when you have new users with few interactions.

from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.metrics.pairwise import cosine_similarity

# Item descriptions
items = [
    "action movie with explosions and car chases",
    "romantic comedy about falling in love",
    "sci-fi thriller in outer space with aliens",
    "action adventure with superheroes saving the world",
    "romantic drama about relationships and family",
    "sci-fi action with robots and AI",
]

# Build item feature matrix using TF-IDF
tfidf = TfidfVectorizer(stop_words='english')
item_features = tfidf.fit_transform(items)

# Find similar items to item 0 (action movie)
similarities = cosine_similarity(item_features[0:1], item_features).flatten()
similar_indices = similarities.argsort()[::-1][1:]  # Exclude itself

print("Most similar to 'action movie with explosions':")
for idx in similar_indices[:3]:
    print(f"  [{similarities[idx]:.3f}] {items[idx]}")

8. Matrix Factorization

Description: Decomposes the sparse user-item rating matrix R into two lower-dimensional matrices: U (user factors) and V (item factors), such that R ≈ U · V^T. Each user and item is represented by a latent factor vector. Unknown ratings are predicted by the dot product of user and item vectors. SVD and ALS are common algorithms.

Use Cases: Netflix Prize, any large-scale recommendation with sparse ratings, latent feature discovery.

import numpy as np

class MatrixFactorization:
    def __init__(self, n_users, n_items, n_factors=20, lr=0.01, reg=0.1):
        self.U = np.random.normal(0, 0.1, (n_users, n_factors))
        self.V = np.random.normal(0, 0.1, (n_items, n_factors))
        self.lr = lr
        self.reg = reg

    def predict(self, user, item):
        return self.U[user] @ self.V[item]

    def fit(self, ratings, n_epochs=100):
        """ratings: list of (user, item, rating) tuples"""
        for epoch in range(n_epochs):
            np.random.shuffle(ratings)
            total_loss = 0
            for user, item, rating in ratings:
                error = rating - self.predict(user, item)
                # SGD update
                self.U[user] += self.lr * (error * self.V[item] - self.reg * self.U[user])
                self.V[item] += self.lr * (error * self.U[user] - self.reg * self.V[item])
                total_loss += error ** 2
            if (epoch + 1) % 20 == 0:
                print(f"Epoch {epoch+1}, Loss: {total_loss:.4f}")

# Example
mf = MatrixFactorization(n_users=5, n_items=5, n_factors=3)
train_data = [(0,0,5),(0,1,3),(0,3,1),(1,0,4),(1,3,1),(2,0,1),(2,1,1),(2,3,5)]
mf.fit(train_data, n_epochs=100)

9. Factorization Machines

Description: A general-purpose model that captures all pairwise feature interactions through factorized parameters. Unlike polynomial regression that needs O(n²) parameters for pairwise interactions, FM uses O(nk) parameters by representing each feature interaction as a dot product of latent vectors. Works well with very sparse, high-dimensional data.

Use Cases: Click-through rate prediction, recommendation with rich side information (user features, item features, context), sparse high-dimensional feature interactions.

import torch
import torch.nn as nn

class FactorizationMachine(nn.Module):
    def __init__(self, n_features, n_factors=10):
        super().__init__()
        self.linear = nn.Linear(n_features, 1)
        self.V = nn.Parameter(torch.randn(n_features, n_factors) * 0.01)

    def forward(self, x):
        # Linear part
        linear_out = self.linear(x)
        # Interaction part (efficient O(nk) computation)
        # sum_of_squares - square_of_sums trick
        square_of_sum = (x @ self.V) ** 2
        sum_of_square = (x ** 2) @ (self.V ** 2)
        interaction = 0.5 * (square_of_sum - sum_of_square).sum(dim=1, keepdim=True)
        return linear_out + interaction

# Example
model = FactorizationMachine(n_features=100, n_factors=10)
x = torch.randn(32, 100)
output = model(x)
print(f"Output shape: {output.shape}")
print(f"Parameters: {sum(p.numel() for p in model.parameters()):,}")