<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0"><channel><title><![CDATA[karthik]]></title><description><![CDATA[In-depth AI, Machine Learning, LLMs, Computer Vision, and GenAI — explained clearly with math, code, and intuition.]]></description><link>https://blog.zynthetix.in</link><generator>RSS for Node</generator><lastBuildDate>Wed, 18 Nov 2026 13:45:46 GMT</lastBuildDate><atom:link href="https://blog.zynthetix.in/rss.xml" rel="self" type="application/rss+xml"/><language><![CDATA[en]]></language><ttl>60</ttl><item><title><![CDATA[Matrix Factorization: The Simplest Deep Explanation You Will Ever Read]]></title><description><![CDATA[A crystal-clear guide to Matrix Factorization with intuition, examples, math, and implementation. Perfect for beginners and AI engineers who want deep…]]></description><link>https://blog.zynthetix.in/matrix-factorization-the-simplest-deep-explanation-you-will-ever-read</link><guid isPermaLink="true">https://blog.zynthetix.in/matrix-factorization-the-simplest-deep-explanation-you-will-ever-read</guid><category><![CDATA[#MachineLearning #Mathematics #RecommenderSystems #DeepLearning #AI #MatrixFactorization #LinearAlgebra]]></category><dc:creator><![CDATA[Gruhesh Sri Sai Karthik Kurra]]></dc:creator><pubDate>Tue, 02 Dec 2025 10:30:59 GMT</pubDate><enclosure url="https://cdn.hashnode.com/res/hashnode/image/upload/v1764671401382/fcaa1362-643b-45a4-9cc4-e5dde61f3ab0.jpeg" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>In notation, we write:</p>
<p>A ≈ U × Vᵀ</p>
<p>But what does this really mean? And why do major AI systems like Netflix, Spotify, Amazon, TikTok, PCA, and SVD rely on it?</p>
<p>Let’s break it down clearly.</p>
<hr />
<h1 id="heading-1-what-matrix-factorization-actually-means"><strong>1. What Matrix Factorization Actually Means</strong></h1>
<p>A matrix is just a big table of numbers:
ratings, pixels, embeddings, features, anything.</p>
<p>Matrix factorization says:</p>
<ul>
<li>The data contains hidden patterns</li>
<li>These patterns are much fewer than the raw dimensions</li>
<li>We can represent the big matrix using two smaller matrices</li>
</ul>
<p>This is called <em>low-rank structure</em>.</p>
<p>Example:</p>
<ul>
<li>A is a large m × n matrix</li>
<li>Choose hidden dimension k (much smaller)</li>
</ul>
<p>We factorize:</p>
<p>U is m × k
V is n × k</p>
<p>Reconstruction:</p>
<p>A ≈ U × Vᵀ</p>
<p>Storage reduces from:</p>
<p>m × n  →  k(m + n)</p>
<p>A massive compression.</p>
<hr />
<h1 id="heading-2-a-simple-real-world-analogy-movie-ratings"><strong>2. A Simple Real-World Analogy: Movie Ratings</strong></h1>
<p>Imagine a large (10,000 × 1,000) user-movie rating matrix.
Most entries are missing.</p>
<p>Matrix factorization assumes:</p>
<ul>
<li>Each user has hidden tastes (action, romance, thrill…)</li>
<li>Each movie has hidden traits (action level, romance level…)</li>
</ul>
<p>So instead of storing the whole matrix, we store:</p>
<ul>
<li>U → user taste vectors</li>
<li>V → movie trait vectors</li>
</ul>
<p>To predict a user’s rating for a movie:</p>
<p>Take the dot product of their vectors.</p>
<p>That’s exactly how Netflix works.</p>
<hr />
<h1 id="heading-3-why-matrix-factorization-works-so-well"><strong>3. Why Matrix Factorization Works So Well</strong></h1>
<p><strong>A. Real-world data is low-rank</strong>
Only a few patterns explain most of the variation.</p>
<p><strong>B. It naturally predicts missing values</strong>
Reconstruction fills them.</p>
<p><strong>C. It denoises</strong>
Only the strongest patterns remain.</p>
<p><strong>D. It compresses dramatically</strong>
Used in LLM weight compression.</p>
<p><strong>E. It speeds up models</strong>
Small matrices → faster operations.</p>
<hr />
<h1 id="heading-4-the-intuition-behind-the-math"><strong>4. The Intuition Behind the Math</strong></h1>
<p>Matrix factorization finds:</p>
<p>U and V such that the difference between A and U × Vᵀ is as small as possible.</p>
<p>Error minimized:</p>
<p>|| A − U × Vᵀ ||²</p>
<p>Gradient descent updates:</p>
<ul>
<li>rows of U</li>
<li>rows of V</li>
</ul>
<p>until the reconstruction error is low.</p>
<p>Think of it like:</p>
<ol>
<li>Discover hidden structure</li>
<li>Store it in U and V</li>
<li>Rebuild the matrix using those hidden factors</li>
</ol>
<hr />
<h1 id="heading-5-one-perfect-example-small-amp-clear"><strong>5. One Perfect Example (Small &amp; Clear)</strong></h1>
<p>Rating matrix:</p>
<pre><code>      M1   M2   M3
U1     <span class="hljs-number">5</span>    ?    <span class="hljs-number">4</span>
U2     <span class="hljs-number">4</span>    <span class="hljs-number">2</span>    ?
U3     ?    <span class="hljs-number">1</span>    <span class="hljs-number">2</span>
</code></pre><p>Assume 2 hidden factors:</p>
<ul>
<li>Action</li>
<li>Romance</li>
</ul>
<p>Let U be:</p>
<pre><code>U =
[
  <span class="hljs-number">0.9</span>   <span class="hljs-number">0.2</span>
  <span class="hljs-number">0.8</span>   <span class="hljs-number">0.6</span>
  <span class="hljs-number">0.1</span>   <span class="hljs-number">0.9</span>
]
</code></pre><p>Let V be:</p>
<pre><code>V =
[
  <span class="hljs-number">0.7</span>   <span class="hljs-number">0.1</span>
  <span class="hljs-number">0.2</span>   <span class="hljs-number">0.8</span>
  <span class="hljs-number">0.9</span>   <span class="hljs-number">0.5</span>
]
</code></pre><p>Predict rating of User 1 for Movie 2:</p>
<p>User 1 vector: 0.9 , 0.2
Movie 2 vector: 0.2 , 0.8</p>
<p>Dot product:</p>
<p>(0.9 × 0.2) + (0.2 × 0.8)
= 0.18 + 0.16
= 0.34</p>
<p>Scaled → roughly 3/5 rating.</p>
<p>This is exactly how recommendation systems work.</p>
<hr />
<h1 id="heading-6-practical-implementation-no-comment-code"><strong>6. Practical Implementation (No-Comment Code)</strong></h1>
<pre><code class="lang-python"><span class="hljs-keyword">import</span> numpy <span class="hljs-keyword">as</span> np

<span class="hljs-function"><span class="hljs-keyword">def</span> <span class="hljs-title">matrix_factorization</span>(<span class="hljs-params">R, k, steps=<span class="hljs-number">5000</span>, lr=<span class="hljs-number">0.0002</span>, reg=<span class="hljs-number">0.02</span></span>):</span>
    m, n = R.shape
    U = np.random.rand(m, k)
    V = np.random.rand(n, k)

    <span class="hljs-keyword">for</span> _ <span class="hljs-keyword">in</span> range(steps):
        <span class="hljs-keyword">for</span> i <span class="hljs-keyword">in</span> range(m):
            <span class="hljs-keyword">for</span> j <span class="hljs-keyword">in</span> range(n):
                <span class="hljs-keyword">if</span> R[i, j] &gt; <span class="hljs-number">0</span>:
                    e = R[i, j] - np.dot(U[i], V[j])
                    U[i] += lr * (e * V[j] - reg * U[i])
                    V[j] += lr * (e * U[i] - reg * V[j])
    <span class="hljs-keyword">return</span> U, V, U @ V.T

R = np.array([
    [<span class="hljs-number">5</span>, <span class="hljs-number">0</span>, <span class="hljs-number">4</span>],
    [<span class="hljs-number">4</span>, <span class="hljs-number">2</span>, <span class="hljs-number">0</span>],
    [<span class="hljs-number">0</span>, <span class="hljs-number">1</span>, <span class="hljs-number">2</span>]
], dtype=float)

U, V, reconstructed = matrix_factorization(R, <span class="hljs-number">2</span>)
print(reconstructed)
</code></pre>
<p>This reconstructs the matrix and predicts missing values.</p>
<hr />
<h1 id="heading-7-final-summary"><strong>7. Final Summary</strong></h1>
<p>Matrix factorization =
low-rank structure + hidden pattern discovery + efficient reconstruction.</p>
<p>It powers:</p>
<ul>
<li>Recommender systems</li>
<li>PCA</li>
<li>SVD</li>
<li>Topic modeling</li>
<li>LLM compression</li>
<li>Image compression</li>
<li>Value prediction</li>
</ul>
<hr />
<h1 id="heading-author"><strong>Author</strong></h1>
<p><strong>Karthik Kurra (Gruhesh Sri Sai Karthik Kurra)</strong>
LinkedIn: <a target="_blank" href="https://www.linkedin.com/in/gruhesh-sri-sai-karthik-kurra-178249227/">https://www.linkedin.com/in/gruhesh-sri-sai-karthik-kurra-178249227/</a>
Portfolio: <a target="_blank" href="https://karthik.zynthetix.in">https://karthik.zynthetix.in</a></p>
<hr />
]]></content:encoded></item><item><title><![CDATA[Understanding Singular Value Decomposition (SVD) — The Cleanest Explanation You Will Ever Read]]></title><description><![CDATA[A clear and beginner-friendly guide to Singular Value Decomposition (SVD). Learn the intuition, formula, geometric meaning, and how SVD powers PCA…]]></description><link>https://blog.zynthetix.in/understanding-singular-value-decomposition-svd-the-cleanest-explanation-you-will-ever-read</link><guid isPermaLink="true">https://blog.zynthetix.in/understanding-singular-value-decomposition-svd-the-cleanest-explanation-you-will-ever-read</guid><category><![CDATA[Machine Learning]]></category><category><![CDATA[ai learning]]></category><category><![CDATA[AI concept]]></category><category><![CDATA[AI]]></category><category><![CDATA[#AIForBeginners ]]></category><dc:creator><![CDATA[Gruhesh Sri Sai Karthik Kurra]]></dc:creator><pubDate>Tue, 02 Dec 2025 09:55:18 GMT</pubDate><enclosure url="https://cdn.hashnode.com/res/hashnode/image/upload/v1764669209393/71cf3d6b-b998-406f-9ad4-9c8c00aa0cad.jpeg" length="0" type="image/jpeg"/><content:encoded><![CDATA[<h1 id="heading-what-is-singular-value-decomposition-svd"><strong>What Is Singular Value Decomposition (SVD)?</strong></h1>
<p>Singular Value Decomposition (SVD) is one of the most powerful ideas in linear algebra and machine learning. It takes any matrix and breaks it into three simple parts:</p>
<p>$$A = U , \Sigma , V^\top$$</p><p>Think of it as:</p>
<p><strong>Rotate → Stretch → Rotate</strong></p>
<p>This gives us the cleanest way to understand what a matrix really does to data.</p>
<hr />
<h1 id="heading-why-should-you-care"><strong>Why Should You Care?</strong></h1>
<p>SVD sits at the heart of almost every major AI/ML workflow:</p>
<ul>
<li><p>Dimensionality reduction (PCA uses SVD internally)</p>
</li>
<li><p>Recommender systems</p>
</li>
<li><p>Image compression</p>
</li>
<li><p>Noise removal</p>
</li>
<li><p>Low-rank optimization (LoRA in large language models)</p>
</li>
</ul>
<p>If you understand SVD, you understand a big part of how modern ML works behind the scenes.</p>
<hr />
<h1 id="heading-1-svd-in-one-sentence"><strong>1. SVD in One Sentence</strong></h1>
<p>SVD breaks a matrix into the simplest possible components: <strong>two rotations and one scaling operation.</strong></p>
<p>This lets you understand the most important patterns inside any dataset.</p>
<hr />
<h1 id="heading-2-intuition-what-does-svd-actually-do"><strong>2. Intuition: What Does SVD Actually Do?</strong></h1>
<p>Imagine a <strong>unit sphere</strong>. When a matrix transforms this sphere, it turns into an <strong>ellipsoid</strong>.</p>
<ul>
<li><p>The <strong>axes</strong> of that ellipsoid tell you <em>where the data stretches the most</em>.</p>
</li>
<li><p>The <strong>lengths</strong> of those axes are the <strong>singular values</strong>.</p>
</li>
<li><p>The <strong>directions</strong> of those axes are the <strong>singular vectors</strong>.</p>
</li>
</ul>
<p><strong>Big singular value → strong directionSmall singular value → weak directionZero singular value → direction collapses (low rank)</strong></p>
<p>This geometric picture is what makes SVD so powerful.</p>
<hr />
<h1 id="heading-3-the-formula-behind-svd"><strong>3. The Formula Behind SVD</strong></h1>
<p>The SVD of a matrix is:</p>
<p>$$A = U , \Sigma , V^\top$$</p><h3 id="heading-u-left-singular-vectors"><strong>U (left singular vectors)</strong></h3>
<p>Basis for output space (rotation)</p>
<h3 id="heading-s-singular-values"><strong>Σ (singular values)</strong></h3>
<p>Strength of each direction (scaling)</p>
<h3 id="heading-v-right-singular-vectors"><strong>Vᵀ (right singular vectors)</strong></h3>
<p>Basis for input space (rotation)</p>
<p>The singular values come from the square roots of the eigenvalues of:</p>
<p>$$A^\top A$$</p><p>and</p>
<p>$$A A^\top$$</p><p>So:</p>
<p>$$\text{Eigenvalues of }(A^\top A) = \sigma_i^2$$</p><hr />
<h1 id="heading-4-where-svd-fits-in-the-aiml-pipeline"><strong>4. Where SVD Fits in the AI/ML Pipeline</strong></h1>
<p>Here is exactly where SVD sits in the workflow:</p>
<hr />
<h3 id="heading-before-svd"><strong>Before SVD</strong></h3>
<ul>
<li><p>Data cleaning</p>
</li>
<li><p>Normalization</p>
</li>
<li><p>Feature scaling</p>
</li>
<li><p>Mean subtraction (for PCA)</p>
</li>
</ul>
<hr />
<h3 id="heading-svd-happens-here"><strong>SVD Happens Here</strong></h3>
<p>Extract structure → find important directions → reveal low-rank patterns.</p>
<hr />
<h3 id="heading-after-svd"><strong>After SVD</strong></h3>
<ul>
<li><p>Dimensionality reduction</p>
</li>
<li><p>Feeding reduced features into ML models</p>
</li>
<li><p>Noise removal</p>
</li>
<li><p>Image compression</p>
</li>
<li><p>Latent factor extraction (recommenders)</p>
</li>
</ul>
<p>Once you see this, SVD is no longer confusing.</p>
<hr />
<h1 id="heading-5-relationship-to-other-concepts"><strong>5. Relationship to Other Concepts</strong></h1>
<h3 id="heading-pca"><strong>PCA</strong></h3>
<p>PCA = run SVD on a centered dataset and select the top-k singular values.</p>
<hr />
<h3 id="heading-eigen-decomposition"><strong>Eigen-decomposition</strong></h3>
<p>For any matrix:</p>
<p>$$\text{Eigenvalues of }(A^\top A) = \sigma_i^2$$</p><p>This links SVD directly to classical eigenvalue methods.</p>
<hr />
<h3 id="heading-lora-in-llms"><strong>LoRA in LLMs</strong></h3>
<p>LoRA fine-tuning works <strong>because weight updates are naturally low-rank</strong>, a property revealed by SVD-like structure.</p>
<hr />
<h3 id="heading-matrix-approximation"><strong>Matrix Approximation</strong></h3>
<p>Truncated SVD gives the <strong>best rank-k approximation</strong> of any matrix:</p>
<p>$$A_k = U_k , \Sigma_k , V_k^\top$$</p><p>No other method can do better (Eckart–Young theorem).</p>
<hr />
<h1 id="heading-6-quick-reflection-31-method"><strong>6. Quick Reflection (3–1 Method)</strong></h1>
<h3 id="heading-3-things-you-learned">✔ <strong>3 Things You Learned</strong></h3>
<ul>
<li><p>SVD = rotate → scale → rotate</p>
</li>
<li><p>Singular values capture the strength of important directions</p>
</li>
<li><p>Truncated SVD = best low-rank approximation</p>
</li>
</ul>
<h3 id="heading-1-place-you-can-apply-this">⭐ <strong>1 Place You Can Apply This</strong></h3>
<p>Use SVD for <strong>PCA</strong>, which gives compact and meaningful features for ML models.</p>
<hr />
<h1 id="heading-final-thoughts"><strong>Final Thoughts</strong></h1>
<p>SVD is not just a mathematical trick — it’s one of the core pillars of modern AI. It extracts structure, compresses data, and helps us understand what really matters inside large datasets and big neural networks.</p>
<p>If you master SVD, you unlock a foundational tool that appears everywhere in machine learning, deep learning, and LLMs.</p>
]]></content:encoded></item></channel></rss>