AI & Machine Learning for Developers
Python for AI and Machine Learning
Most of the Python you use for AI and machine learning work is not new syntax — it is a small set of libraries and habits for handling numeric and tabular data. Arrays (NumPy) and tables (pandas) replace the lists and dictionaries you would reach for in general-purpose code, because models expect data in a specific numeric shape. This guide assumes you already know core Python (variables, loops, functions) and focuses on the data layer that sits between raw data and a model.
Why it matters
- Data prep is most of the actual work
- Cleaning, reshaping, and checking data typically takes far longer than writing or calling the model itself, so the tools for doing it well matter more than they first appear to.
- Vectorized operations replace explicit loops
- NumPy and pandas push repetitive work into fast, pre-compiled routines, so idiomatic code applies an operation to a whole array or column at once instead of looping row by row.
- The same array shape underlies everything downstream
- A NumPy array's shape (rows, columns, dimensions) is the same concept behind a pandas DataFrame, a scikit-learn feature matrix, and a PyTorch or TensorFlow tensor, so learning to reason about shape once pays off across all of them.
- Notebooks change how you write and check code
- Jupyter notebooks let you run one cell at a time and look at the data after each step, which suits exploratory data work better than writing a whole script blind.
- Small data mistakes cause large, silent model problems
- A wrong join, a column read as text instead of a number, or a leaked test row rarely throws an error — it just quietly produces a worse or meaningless model.
Arrays and vectorized thinking
NumPy's core idea is the array: a grid of numbers, all of the same type, with a fixed shape. Once data is in an array, an operation like multiplying by a constant or comparing to a threshold applies to every element at once, without writing a loop yourself. This matters because AI/ML code is read and reviewed constantly, and "multiply the whole column by 1.08" is both shorter and closer to the actual intent than a for loop with an index variable. The shape of an array — how many rows, how many columns, how many dimensions — is worth being deliberate about, because most errors in this area are shape mismatches: an operation expecting a table gets a single row, or two arrays that should line up don't.
import numpy as np
prices = np.array([19.99, 5.50, 42.00])
# Vectorized: the multiplication applies to every element at once
prices_with_tax = prices * 1.08
print(prices_with_tax)
# [21.5892 5.94 45.36 ]Tables of data: the DataFrame
pandas builds on NumPy's arrays to add labeled rows and columns, which is closer to how real data actually arrives — a spreadsheet or a database export, not a bare grid of numbers. A DataFrame lets you select a column by name, filter rows by a condition, and summarize a group of rows (a groupby, such as "average order value per customer") in a line or two. Missing values are a normal, expected part of this work rather than an edge case: real datasets have gaps, and deciding whether to drop, fill, or flag a missing value is a judgment call the library will not make for you.
import pandas as pd
df = pd.read_csv("orders.csv")
# Rows where the order was over 100, one column
large_orders = df[df["total"] > 100]["customer_id"]
# Average order value per customer
average_by_customer = df.groupby("customer_id")["total"].mean()Getting data into the shape a model expects
Models generally do not accept raw text, dates, or category labels directly — they expect numbers, arranged in a consistent shape. Turning a category like "country" into numbers (encoding), turning free text into numbers (covered in this site's embeddings guide), and turning a date into separate numeric fields are all instances of the same underlying step. It is also standard practice to hold out part of the data — commonly by splitting it into a training portion and a separate test portion before any modeling starts — so a model's score is not just a report of how well it memorized data it has already seen. Which specific split or encoding to use is a modeling decision beyond this guide's scope; what matters here is recognizing that this conversion step exists and has to happen before the model sees anything.
Mistakes people make here
- Looping over a DataFrame row by row
- It works, but it is usually both slower and a sign that a built-in vectorized method already does the same job in one line — pandas and NumPy are built around avoiding exactly this pattern.
- Not checking for missing values before modeling
- Many libraries either error out on a missing value or silently treat it in a way you didn't intend; checking for and deciding how to handle gaps is a required step, not an optional cleanup.
- Mistaking a copy of a DataFrame for a view of the original
- pandas sometimes returns a view and sometimes a copy depending on the operation, so an edit meant to be local can silently change the original data — this is common enough that pandas has a specific warning for it.
- Evaluating a model on the same data it was trained on
- This is fundamentally a data-handling mistake: if the "test" data was also seen during training, the resulting score measures memorization, not the model's ability to generalize to new data.
- Treating a shape-mismatch error as a basic Python bug
- Messages about a dimension or shape mismatch come from the array-shape model underneath NumPy and pandas, not from core Python syntax, so debugging them means checking shapes and dtypes, not re-reading loop logic.
Strengths and trade-offs
Where it is strong
- A small vocabulary — select, filter, group, join, reshape — covers the large majority of real data-preparation work.
- Vectorized code is usually both shorter and closer to the actual intent ("scale this column") than the equivalent loop.
- NumPy arrays are the common currency across the ecosystem: they flow into pandas, scikit-learn, and deep learning frameworks without conversion.
- The libraries are open source, widely documented, and stable enough that code written years ago mostly still runs.
The trade-offs
- The vectorized style takes real practice to think in; "avoid the loop" is not the instinct most general-purpose programming teaches.
- Data problems like a wrong join key or a mis-typed column often don't raise an error — they just quietly degrade the result.
- Datasets too large to fit in memory need a different set of tools (chunked processing, or a database) than a standard pandas workflow assumes.
Who needs this
Anyone whose work touches a model's data — training one, evaluating one, or building the pipeline that feeds one — needs this layer, even without ever writing a training loop. If you only call an already-trained model through an API and never look at its input data yourself, you can treat this as background knowledge rather than a daily tool.
Questions about python for ai and machine learning
- Do I need to learn NumPy and pandas as two separate things?
- It helps to know that pandas is built on top of NumPy's arrays, so the same idea of a fixed shape applies to both. In practice most day-to-day data work happens in pandas, with NumPy showing up underneath or for lower-level numeric operations.
- Is this guide teaching me to train a model?
- No. This is the data-handling layer that sits before and around modeling — how data gets cleaned, shaped, and split. Training and evaluating a model is a broader topic covered conceptually in this site's machine learning fundamentals guide.
- Do I need a powerful computer to practice this?
- No. The concepts — arrays, DataFrames, filtering, grouping — are the same on a small sample dataset as on a large one; only the performance characteristics change with size.
- How does this relate to the site's core Python guide?
- It assumes the fundamentals from that guide — variables, loops, functions, data types — and adds the specific libraries and habits used once you're handling data for a model rather than general-purpose scripting.