Pandas Interview Questions

A Series is a single column of data, a one-dimensional labeled array. A DataFrame is a full table, two-dimensional, made up of multiple Series sharing the same index. If you pull one column out of a DataFrame, you get back a Series.

import pandas as pd. The pd alias is a near-universal convention, so much so that seeing pandas imported under any other name in someone else's code tends to slow a reader down for a second, wondering why they broke from the norm.

Numeric types like int64 and float64, object (usually text, though it can hold mixed types), bool, datetime64 for dates and times, and category for a column with a limited set of repeating values. Picking the right dtype matters for both correctness and memory usage, especially on a large dataset.

df.shape gives you a tuple of (rows, columns). df.info() shows column names, their dtypes, and how many non-null values each has. df.head() shows the first few rows so you can eyeball what the data actually looks like before doing anything else with it.

head() shows the first n rows of a DataFrame, defaulting to 5 if you don't specify a number. tail() shows the last n rows instead. Checking both is a quick habit worth building, since a dataset can look fine at the top and have obvious problems, like a stray footer row, at the bottom.

pd.DataFrame({'name': ['Anu', 'Ben'], 'age': [25, 30]}). Each key becomes a column name, and each list becomes that column's values. It's one of the most common ways to build a small DataFrame directly in code, especially for testing or a quick example.

pd.read_csv('filename.csv'). It's one of the most-used functions in the entire library, and it comes with a long list of optional parameters for handling different delimiters, missing value markers, and header rows, since real-world CSV files rarely arrive in a perfectly clean format.

Pass the usecols parameter to read_csv, like pd.read_csv('file.csv', usecols=['name', 'age']). This is especially useful for a wide file with dozens of columns where you only actually need a handful, since it avoids loading and holding unnecessary data in memory.

df.to_csv('output.csv', index=False). The index=False part matters. Without it, Pandas writes the DataFrame's index as an extra unnamed column in the output file, which is rarely what you actually want.

Excel (read_excel), JSON (read_json), SQL databases (read_sql), and Parquet (read_parquet) are the common ones. Parquet in particular is worth knowing about early, since it's a columnar format that's both smaller on disk and faster to read than CSV for large datasets.

df.columns gives you just the column names. df.dtypes gives you the data type of each column. Both are quick, lightweight checks you'd run right after loading a new dataset, before diving into df.info() for the fuller picture.

loc selects rows and columns by label, the actual index value or column name. iloc selects by integer position, regardless of what the labels actually are. df.loc[0] finds the row labeled 0, which is usually, but not always, the first row. df.iloc[0] always means the first row, no matter what it's labeled.

df['column_name'] returns that column as a Series. df.column_name works too, as long as the column name is a valid Python identifier with no spaces or special characters, though the bracket syntax is generally the safer habit to build.

Pass a list of column names inside the brackets, df[['name', 'age']]. Note the double brackets, the outer one is the DataFrame's indexing syntax, and the inner one is the actual list of columns you want.

Boolean indexing filters rows using a condition that evaluates to True or False for each row. df[df['age'] > 30] returns only the rows where the age column is greater than 30. Under the hood, the condition itself produces a Series of True/False values, and the DataFrame keeps only the rows lined up with True.

Combine conditions with & for and, or | for or, wrapping each individual condition in its own parentheses. df[(df['age'] > 30) & (df['city'] == 'Mumbai')] returns rows matching both conditions. Using Python's plain and/or keywords here instead of & and | is a common mistake that throws a confusing error.

df.loc[row_label, 'column_name'] returns that single value directly. df.loc[0:5, ['name', 'age']] returns a slice of rows and a specific set of columns together, which is one of the more common patterns once you're regularly slicing data both ways at once.

df['new_column'] = some_value_or_series. If you assign a single value, every row in that new column gets that same value. If you assign a Series or a computed expression, like df['total'] = df['price'] * df['quantity'], each row gets its own calculated result.

df.drop('column_name', axis=1) drops a column, since axis=1 refers to columns while axis=0 refers to rows. Passing inplace=True modifies the DataFrame directly rather than returning a new one, which matters since forgetting it is a common source of a drop that silently doesn't seem to take effect.

df.sort_values('column_name') sorts in ascending order by default. Passing ascending=False reverses that. You can sort by multiple columns at once by passing a list of column names, which sorts primarily by the first and uses the rest as tie-breakers.

df.rename(columns={'old_name': 'new_name'}). You can rename several columns at once by adding more key-value pairs to that same dictionary, and like drop, it returns a new DataFrame unless you pass inplace=True.

df['column'].apply(some_function) runs that function on each value in the column and returns a new Series with the results. For simple transformations, a built-in vectorized method is usually faster than apply, but apply is the right tool once the logic is too custom for a built-in method to express directly.

df['column'].describe() returns count, mean, standard deviation, min, max, and the quartiles all at once. Calling it on the whole DataFrame instead of one column runs the same summary across every numeric column simultaneously.

df.isna() (or the equivalent df.isnull()) returns a same-shaped DataFrame of True/False values marking where data is missing. Chaining .sum() on top of that, df.isna().sum(), gives you a quick count of missing values per column, which is usually the first thing worth checking on a new dataset.

df.dropna() removes any row that has at least one missing value by default. Passing how='all' only drops a row if every single value in it is missing, and passing a subset parameter restricts the check to specific columns rather than the whole row.

df.fillna(value) replaces every missing value with the value you provide. That value can be a single number or string applied everywhere, or a dictionary specifying a different fill value for each column, depending on how uniform the fix actually needs to be.

Dropping loses the entire row (or column), which is fine when missing data is rare and random, but risky when it's common or concentrated in a way that would bias the remaining data. Filling keeps every row but introduces an assumption about what the missing value should have been, a mean, a zero, a forward-filled previous value, so the right choice depends on how much data you can afford to lose versus how comfortable you are with that assumption.

df.fillna(method='ffill') (or the newer df.ffill()) carries the last valid value forward into each subsequent missing value. This makes sense for something like a daily stock price where a missing day probably means no trading happened, not that the price actually vanished.

groupby splits a DataFrame into groups based on the values in one or more columns, so you can then run an aggregation, like sum, mean, or count, separately for each group. df.groupby('city')['sales'].sum() gives you total sales per city in one line.

df['column'].value_counts() returns the count of each unique value, sorted from most to least common by default. It's one of the fastest ways to get a quick sense of how a categorical column is actually distributed.

df.groupby('category')['price'].mean(). This groups the DataFrame by the category column, then calculates the average price within each group separately, returning one mean value per unique category.

df['column'].nunique() returns the count of distinct values, ignoring duplicates. df['column'].unique() returns the actual distinct values themselves as an array, useful when you want to see what the values are, beyond just how many of them there are.

df.groupby('category')['price'].agg(['mean', 'sum', 'count']) runs all three aggregations at once on the same grouped data, returning a DataFrame with one column per aggregation, rather than calling groupby three separate times for three separate results.

sum() collapses a column down to a single total number. cumsum() returns a running total instead, one value per row, where each row's value is the sum of everything up to and including that row. cumsum() is what you'd reach for to build something like a running balance.

First compute the total per group with groupby and transform, then divide each row's value by that broadcast group total. df['pct'] = df['sales'] / df.groupby('region')['sales'].transform('sum') gives every row its own share of its region's total, without needing a separate merge step to bring the group total back onto each row.

filter() on a grouped object keeps or drops entire groups based on a condition evaluated on the whole group, like keeping only categories with more than 100 rows. A regular boolean filter operates row by row instead, with no awareness of which group a row belongs to.

Pass a list of column names to groupby, df.groupby(['region', 'category'])['sales'].sum(). This creates a hierarchical grouping, giving you a sum for every unique combination of region and category rather than for each column independently.

Named aggregation lets you apply different functions to different columns after a groupby and control the resulting column names directly, using df.groupby('category').agg(avg_price=('price', 'mean'), total_qty=('quantity', 'sum')). Without it, the default output of a multi-column agg() call often produces column names that need renaming afterward anyway.

merge() combines two DataFrames based on matching values in a shared column, similar to a SQL join. concat() simply stacks DataFrames together, either on top of each other (rows) or side by side (columns), without matching on any key at all.

inner keeps only rows with matching keys in both DataFrames. left keeps all rows from the left DataFrame, filling in missing values where there's no match on the right. right does the same but keeps everything from the right DataFrame instead. outer keeps every row from both sides, filling in missing values wherever a match doesn't exist.

Use left_on and right_on instead of the single on parameter, pd.merge(df1, df2, left_on='customer_id', right_on='cust_id'). This tells Pandas which column to match on each side, since the default on parameter assumes both sides share the exact same column name.

Pandas automatically appends suffixes, _x and _y by default, to distinguish the two versions of that column in the merged result. You can control those suffixes explicitly with the suffixes parameter if the defaults aren't descriptive enough for your case.

Check df['key_column'].duplicated().sum() on each DataFrame before merging. If the join key isn't unique on one or both sides, a merge can produce far more rows than either original DataFrame had, since every matching combination gets its own row, a surprise that's much easier to catch beforehand than to debug afterward.

df.duplicated() returns a boolean Series flagging duplicate rows. df.drop_duplicates() removes them outright, keeping the first occurrence by default. Passing subset restricts the duplicate check to specific columns rather than requiring every column to match exactly.

df['column'] = df['column'].astype('int64') converts a column to the specified type directly. For dates specifically, pd.to_datetime(df['column']) handles a wider range of date formats and gives clearer errors than a generic astype() call, which is why it's the safer default for date conversion specifically.

Strip out the non-numeric characters first with string methods, then convert. df['price'] = df['price'].str.replace('$', '').str.replace(',', '').astype(float) is a typical pattern, chaining string cleanup before the final type conversion.

Accessed through the .str accessor, like df['name'].str.lower() or df['name'].str.contains('a'), these apply a string operation to an entire column at once, implemented in optimized code under the hood. They're dramatically faster than looping through rows in plain Python for anything beyond a tiny dataset.

A mapping dictionary passed to .replace(), or .str.lower().str.strip() combined with a lookup table for the remaining variants, depending on how many distinct spellings actually exist. For a small, known set of variants, an explicit mapping is usually clearer and easier to audit than a fuzzy-matching approach.

Both reshape data from a long format into a wider one, turning unique values from one column into new columns. pivot() requires the combination of index and columns to be unique, with no duplicates, and will raise an error otherwise. pivot_table() handles duplicates by aggregating them automatically, using a function like mean or sum, which is why it's the more commonly used of the two in practice.

melt() reshapes a wide DataFrame into a long one, turning multiple columns into rows with a variable name and value column. It's the reverse of a pivot, and you'd reach for it when data arrives with, say, a separate column per month, and you actually need one row per month instead for further analysis or plotting.

stack() moves the innermost column level into the row index, making the DataFrame taller and narrower. unstack() does the reverse, moving the innermost row index level into columns, making it wider and shorter. Both work specifically with a DataFrame's index and column levels, which matters most once you're dealing with a MultiIndex.

melt() is usually the right tool, since most plotting libraries expect long-format data, one row per observation, rather than one column per category. Converting to long format first, then plotting, is a far more common workflow than trying to plot directly from a wide DataFrame.

pd.to_datetime(df['date_column']). Once converted, the column supports date-specific operations, extracting the year or month, calculating the difference between two dates, filtering by a date range, none of which work correctly on plain strings.

df.set_index('date_column'), after first converting that column with pd.to_datetime() if it isn't already a datetime type. A datetime index unlocks time-based slicing, like df['2026-01':'2026-03'], and is a prerequisite for resampling.

resample() groups data by a time frequency, daily, weekly, monthly, rather than by the value in a regular column. df.resample('M').sum() gives you a monthly total, treating the DataFrame's datetime index as the grouping key automatically, something a regular groupby() has no built-in concept of.

df['column'].rolling(window=7).mean(). This computes the average over each trailing 7-row window, sliding forward one row at a time, commonly used to smooth out day-to-day noise in a time series before looking at the underlying trend.

Downcast numeric columns to smaller dtypes where the data allows it, int64 down to int32 or even int8 if the values fit, float64 down to float32 if the extra precision isn't needed. Converting a column with a small number of repeating string values to the category dtype can also cut memory dramatically, since Pandas then stores each unique value once instead of repeating the full string on every row.

It stores a column's distinct values once and represents each row as a reference to one of those values, rather than repeating the full string every single time. It's a strong fit for a column like a country name or a status flag, where a small set of values repeats across many rows, cutting memory use significantly and often speeding up groupby operations on that column too.

Specify dtypes explicitly with the dtype parameter, so Pandas doesn't have to infer them column by column. Read only the columns you actually need with usecols. For files too large to fit comfortably in memory at all, reading in chunks with chunksize processes the file piece by piece instead of loading everything at once.

iterrows() returns each row as a Series, which is flexible but has real overhead from constructing that Series object every single row. itertuples() returns each row as a lightweight named tuple instead, which is meaningfully faster than iterrows() but still far slower than a vectorized operation. Both exist for the cases where a genuinely row-by-row operation is unavoidable, not as a general-purpose way to process a DataFrame.

The %timeit magic command in a Jupyter notebook gives a quick benchmark for a single line or operation. For a longer, more complex pipeline, a proper profiler like cProfile, or simply timing each step manually and printing the results, usually reveals that one specific step, often an unnecessary loop or a poorly chosen merge, accounts for most of the total time.

Copy-on-write changes how Pandas handles operations that used to sometimes modify data in place unpredictably, making chained operations behave more consistently and eliminating a lot of the confusing SettingWithCopyWarning cases that used to trip people up. It was introduced to make Pandas' behavior more predictable, at the cost of some existing code that relied on the old implicit in-place behavior needing adjustment.

A MultiIndex is a hierarchical index made up of multiple levels, letting you represent higher-dimensional data, like sales broken down by both region and year, within a standard two-dimensional DataFrame. It's what you typically get automatically after grouping by more than one column.

df.loc[('North', 2026)] selects using a tuple matching each level of the index in order. df.xs('North', level='region') selects a cross-section based on just one specific level, which is often more convenient when you don't need to specify every level at once.

It converts the current index (including every level of a MultiIndex) back into regular columns, replacing it with a fresh default integer index. It's commonly used right after a groupby operation, since the grouped column becomes the index by default, and reset_index() turns it back into a normal column for further merging or display.

swaplevel() swaps exactly two specified levels of the index. reorder_levels() lets you rearrange all the levels into any order you specify at once, which matters when you're working with three or more index levels and need more than a simple two-way swap.

df.sort_index() sorts by the index itself, respecting the hierarchy level by level. df.sort_index(level='year') sorts specifically by one named level rather than the full hierarchy. Note that many MultiIndex operations, including certain kinds of slicing, actually require the index to be sorted first or they'll raise an error.

df.columns = ['_'.join(col).strip() for col in df.columns.values] joins each tuple of column levels into one flat string, so a column that was ('price', 'mean') becomes 'price_mean'. This is a common cleanup step right after a pivot_table call, since downstream code and most plotting libraries expect flat, single-level column names.

Method chaining links several DataFrame operations together in one expression, df.dropna().groupby('category').mean().reset_index(), instead of assigning an intermediate variable after each step. It reads cleanly once you're used to the style, but a chain that's too long can be genuinely hard to debug, since there's no intermediate variable to inspect when something in the middle goes wrong.

df.apply(function, axis=1) passes each entire row to your function as a Series, letting you reference multiple columns from within it. It carries the same per-row overhead as any other apply() call, so for anything performance-sensitive, expressing the same logic as a vectorized combination of columns is worth the extra effort if it's genuinely possible.

Define a plain Python function that takes a Series and returns a single value, then pass it directly into agg(), df.groupby('category')['price'].agg(lambda x: x.max() - x.min()). This is how you'd calculate something groupby doesn't provide as a built-in, like a custom range or a weighted average.

df['column'].map(mapping_dict) is typically the fastest built-in option for a dictionary-based lookup, since it's implemented as a vectorized operation rather than a Python-level loop. For a mapping that involves a range condition rather than an exact match, pd.cut() or a vectorized comparison usually beats writing a custom apply() function.

df.copy(deep=False) creates a new DataFrame object, but its underlying data still shares memory with the original, so a change to one can unexpectedly affect the other. df.copy(deep=True), which is the default, fully duplicates the underlying data too, so the two DataFrames become genuinely independent. This distinction is exactly what's behind Pandas' notorious SettingWithCopyWarning.

A Pandas DataFrame is built on top of NumPy arrays internally, which is exactly why vectorized Pandas operations run as fast as they do, they're often calling straight into NumPy's compiled operations underneath. Understanding this connection helps explain why certain operations that break vectorization, like a Python-level apply(), lose that performance benefit entirely.

Read and process it in chunks using the chunksize parameter in read_csv(), handling each chunk independently and aggregating results as you go rather than loading the whole file at once. For datasets that are consistently too large for this approach to be practical, a tool built for out-of-core or distributed processing, like Dask or Polars, is usually the better fit.

Dask provides a Pandas-like API that operates on data too large to fit in memory, or spread across multiple machines, by breaking it into partitions and processing them lazily and in parallel. Code written for Dask looks very similar to Pandas code, which lowers the barrier to scaling up an existing Pandas-based workflow without a full rewrite.

Polars is a DataFrame library built in Rust with a focus on performance, using a fully lazy query execution model that can optimize an entire chain of operations before running any of them, unlike Pandas' typically eager, step-by-step execution. For large datasets, it can be dramatically faster, though its API, while similar in spirit to Pandas, differs enough that switching isn't always a drop-in replacement.

A schema validation library like Pandera or Great Expectations lets you declare expected column types, value ranges, and null constraints, then checks incoming data against those rules automatically before it ever reaches your actual processing logic. Catching a schema violation at the door is far cheaper than discovering it three transformations deep in a pipeline where the root cause has become much harder to trace.

pd.read_sql() reads a query's results directly into a DataFrame, and df.to_sql() writes a DataFrame back into a database table. For anything beyond a small dataset, batching the writes (using the chunksize parameter on to_sql) avoids overwhelming the database with one enormous insert statement.

If the source is already a database, pushing filtering and aggregation into the SQL query itself is usually faster, since the database can use its own indexes and avoid pulling unnecessary rows across the network in the first place. Pandas becomes the better fit for the transformations that are awkward or impossible to express cleanly in SQL, complex row-wise logic, integration with other Python libraries, or genuinely exploratory analysis.

Standardize each source's column names right after loading it, before combining anything, using a rename mapping specific to each source. Combining first and trying to reconcile inconsistent names afterward tends to create a much messier cleanup step than fixing each source individually up front.

I'd favor clear, well-named intermediate steps over a single dense, heavily-chained transformation, even if the chained version runs marginally faster, since a pipeline someone else has to debug at 2am benefits far more from readability than from shaving a few milliseconds off. Validating data at each major stage, rather than only at the very end, also makes it dramatically easier to find exactly where something went wrong.

I look at how it handles bad or unexpected input, since real-world data eventually sends something malformed no matter how careful the upstream systems are, and the happy path alone tells you nothing about that. I also check whether the pipeline is idempotent, safe to rerun without duplicating or corrupting data, since a pipeline that isn't will eventually need to be rerun after a failure, and that's exactly the wrong moment to discover it can't be.

I'd push for automated schema and data quality checks (via something like Pandera or Great Expectations) built into the pipeline itself, rather than relying on people manually eyeballing output, since manual checks get skipped the moment someone's in a hurry. Documenting the handful of data quality rules that actually matter for a given dataset, with the reasoning behind them, spreads faster than a long, generic data governance document nobody reads in full.

I'd weigh the real performance and memory benefits for the specific workloads that are actually struggling today, against the cost of retraining a team on new syntax, and the practical reality that most existing Pandas code and its many dependent libraries won't migrate overnight. A gradual adoption on new, performance-sensitive pipelines, while leaving stable existing pipelines alone, usually beats a wholesale, disruptive switch.

I'd suspect the data before the code. Intermittent incorrect results in a pipeline that hasn't changed usually trace back to a change in the upstream data itself, a new category value appearing, a date format shifting, a source system starting to send occasional nulls where it never used to. I'd add data validation checks at the pipeline's entry point specifically to catch this category of problem going forward, rather than only patching the one instance that happened to get noticed.

Track row counts and key aggregate statistics at each major stage, and alert when they drift unexpectedly outside a normal range. An outright error is the easy case to catch. A pipeline that runs successfully but silently drops half its rows due to an unexpected join mismatch is a genuinely worse failure than one that crashes loudly, since nobody notices until someone downstream asks why a number looks wrong.

Treat the output schema as a contract with everyone downstream. Adding a new column is generally safe. Renaming or removing an existing column needs a deprecation period and direct communication with whoever's consuming it, since a silent schema change can break another team's pipeline in a way that's genuinely hard for them to trace back to your change.

Mitigate first: flag the report as under review so nobody makes a decision based on numbers that might be incorrect, before digging into root cause. Then I'd trace it systematically from the output backward through each transformation step, checking assumptions about the input data at each stage, since a wrong aggregation is far more often caused by unexpected upstream data than by an actual bug in stable, previously-working transformation logic.

Start from an actual test with a realistically scaled-up sample of data rather than a rough guess based on current volume. I'd specifically check where memory usage spikes during the pipeline, since a large merge or a wide pivot can multiply memory usage well beyond what the input data size alone would suggest, and that's usually the actual constraint that shows up first, before raw processing time does.

I'd weigh engineering time against infrastructure cost honestly. If a pipeline runs once a week and a bigger machine for an hour costs a few dollars, that's almost always cheaper than the days it would take an engineer to optimize it. Optimization is worth the investment once a pipeline runs frequently enough, or is central enough to other work, that the recurring cost, in either compute or people waiting on it, actually adds up to something significant over time.

I'd take one of their actual slow scripts, time it together, then rewrite the same logic as a vectorized operation right in front of them, showing the concrete speedup rather than explaining vectorization as an abstract concept. Seeing their own five-minute script finish in two seconds tends to change how they approach the next problem far more than a general rule about avoiding loops ever does.

I wouldn't lead with process for its own sake. I'd point to a specific incident where an untested notebook script produced a wrong number that reached a stakeholder, and show how a basic test or validation check would have caught it before it went out. A concrete, already-felt consequence is far more persuasive than an argument for best practices in the abstract.

I'd bring the actual performance numbers and data volume behind my position, rather than a general preference for one tool over the other. Most disagreements like this resolve once both sides are looking at the same concrete numbers about where the actual data lives and how much of it needs to move across the network to get processed.

I'd translate the risk into terms leadership already tracks: the specific incident history of that script breaking or producing wrong numbers, the hours spent firefighting it each time that happens, and what a repeat of the worst version of that failure would actually cost the business. Framed as risk reduction with real, already-incurred cost behind it, it competes far better for engineering time than framed as a general code-quality improvement.