Managing Data

A vague prompt like this produces unreliable output:

Tell me about the penguins dataset.

The LLM answers from its training data, not from the file in front of you. It may describe a version of the dataset that differs from yours, invent column names or units that do not exist, or report statistics that do not match your file. The answer will sound authoritative but you cannot verify any of it without checking the original source yourself, at which point you did not need the LLM. Fix it by pasting the actual file contents into the prompt.

I am pasting the first five lines of a CSV file below. List each column name exactly as it appears in the header, infer its most likely data type (integer, float, or string) and units if any, write a one-sentence description of what the column measures, and flag any column whose purpose is not obvious from its name. Do not add any columns that are not in the file.

species,island,bill_length_mm,bill_depth_mm,flipper_length_mm,body_mass_g,sex Adelie,Torgersen,39.1,18.7,181,3750,MALE Adelie,Torgersen,39.5,17.4,186,3800,FEMALE Adelie,Torgersen,40.3,18.0,195,3250,FEMALE Adelie,Torgersen,NA,NA,NA,NA,NA

A typical response:

The LLM also notes that the fourth data row is all NA, which suggests there are rows with entirely missing measurements that may need to be dropped.

Start with the column names. Run head -1 _extras/penguins.csv and compare every name in the LLM's table to the actual header one by one. A hallucinated column name, or one that was silently corrected, means the dictionary cannot be trusted.

Next, check the value ranges. Run cut -d, -f3 _extras/penguins.csv | sort -n | head -3 to see the smallest bill lengths and cut -d, -f3 _extras/penguins.csv | sort -n | tail -3 to see the largest. If the LLM claimed bill length is "typically between 35 and 60 mm", confirm those numbers match the file.

Finally, check the missing value claim. Run grep -c "NA" _extras/penguins.csv and compare the count to the LLM's description of how missing values are encoded. If the LLM said there are two rows with entirely missing values but grep finds NA scattered across many rows, the explanation is wrong.

A prompt like this is too vague:

How do I look at a CSV file?

The LLM will suggest opening it in a spreadsheet, loading it with Python, or using a text editor. None of those report the file's encoding before you have committed to opening it, and none can be scripted into a reproducible workflow. Fix it by naming the file and the specific properties you want to know.

Write and run three shell commands for _extras/squirrels.csv: one to report its encoding, one to count the number of lines, and one to show the first three rows.

A typical response:

file _extras/squirrels.csv
wc -l _extras/squirrels.csv
head -3 _extras/squirrels.csv

Running these shows the encoding (UTF-8 Unicode text), the total line count, and the header plus the first two data rows.

Compare the line count from wc -l to the count your collaborator or the dataset documentation claims. The squirrels census documentation reports 3023 sightings; wc -l returns one more because it counts the header row. If file reports Latin-1 instead of UTF-8, the file may contain characters outside the ASCII range that will raise errors when you load it into Python.

A prompt like this is incomplete:

Count the hectares.

The LLM does not know which file, which column, or which tool you want. It may write Python when you want a shell one-liner, or it may include the header row in the count. Fix it by naming the file, the column position, and the tools.

Write and run a shell pipeline that counts the number of distinct values in column 4 of _extras/squirrels.csv, excluding the header row.

A typical response:

tail -n +2 _extras/squirrels.csv | cut -d, -f4 | sort -u | wc -l

Running this returns 339.

The official squirrels census documentation says there are 339 hectares. If your count is 340, the header was not excluded despite tail -n +2. If it is 338, run tail -n +2 _extras/squirrels.csv | cut -d, -f4 | sort -u | head -5 to inspect the first few values for inconsistent formatting that might split one logical hectare into two distinct strings, or tell the LLM that its answer is wrong, explain how it is wrong, and prompt it to try again.

Write the sqlite3 commands to list all tables in _extras/example.db and display the full schema of the penguins table.

A typical response:

sqlite3 _extras/example.db ".tables"
sqlite3 _extras/example.db ".schema penguins"

The schema shows column types such as real and integer that the CSV header does not provide.

Compare every column name in the schema to the CSV header from head -1 _extras/penguins.csv; they should match exactly, including capitalization and underscores. Then check the declared types against the values you see in the file: if flipper_length_mm is declared as integer in the schema but the CSV contains decimal values in some rows, queries that filter on that column may behave unexpectedly.

A prompt like this isn't useful:

Can I use this dataset in my paper?

The LLM does not know which dataset, or whether your concern is the license, the file permissions, the format, the content, or something else entirely. Fix it by naming the license and describing your intended use in specific terms.

The Palmer Penguins dataset is released under a CC0 1.0 Universal license. Does this license allow me to use the data in a published paper, include it as a downloadable supplement, and publish a cleaned version as a separate dataset?

A typical response explains that CC0 releases data into the public domain, allowing all three uses without any attribution requirement, though citing the original authors remains good practice.

Read the actual CC0 license at https://creativecommons.org and confirm the LLM's summary for each use case. Then ask: if the dataset were CC-BY-NC instead of CC0, which of your three uses would be restricted? Verify that answer against the CC-BY-NC license text as well.

A prompt like this cannot be answered:

Is the squirrels dataset up to date?

"Up to date" means nothing without knowing what question you are trying to answer. Fix it by stating your research question and asking whether the dataset's collection date affects your ability to answer it.

The Central Park squirrel census data was collected in October 2018. I want to study whether squirrel fur color distribution has changed over time. Is a single year of data from 2018 adequate, and if not, what data would I need?

A typical response explains that a trend question requires at least two comparable time points, and notes that an earlier squirrel census was conducted in 2012 using similar methods.

Verify the claim about the 2012 census independently before designing a study around it. If no 2012 census exists, or its methodology is too different to compare, the LLM's suggestion doesn't help you. Specific-sounding claims about the existence of external datasets are exactly what LLMs hallucinate most confidently.

A prompt like this produces a generic template rather than a useful record:

Write a log entry for my squirrel analysis.

The LLM doesn't know what details you want recorded, so it will invent plausible-sounding steps. Fix it by giving the LLM the specific facts and asking it only to format them.

I asked you to write a shell command to count squirrel sightings by primary fur color. The output showed gray as the most common color. Write a log entry based on our conversation in this session with five fields: date, prompt sent, tool used, output kept (yes/no), corrections made.

A typical response produces a structured entry with all five fields, recording the exact prompt text and the specific correction rather than a paraphrase.

Confirm the log entry contains the exact prompt you sent, not a reworded version. (Remember, LLMs can hallucinate about themselves…) If you can, ask a colleague whether they could reproduce your steps from the log entry alone.

A prompt like this is too vague to produce a useful assertion:

Make my data loading script safer.

"Safer" could mean any number of things. Fix it by naming the specific conditions you want to catch and the exact expected values.

I have a Polars script that loads _extras/penguins.csv. Add two assertions after loading: one that raises an error if the number of columns is not 7, and one that raises an error if the number of rows is fewer than 340. Print the actual shape before checking.

A typical response:

import polars as pl

df = pl.read_csv("_extras/penguins.csv")
print(f"Shape: {df.shape}")
assert df.shape[1] == 7, f"Expected 7 columns, got {df.shape[1]}"
assert df.shape[0] >= 340, f"Expected at least 340 rows, got {df.shape[0]}"

Run the script against the real file and confirm both assertions pass. Then verify the column assertion fires when it should: run cut -d, -f1-5 _extras/penguins.csv > /tmp/short.csv, change the filename in the script to /tmp/short.csv, and confirm the script raises AssertionError with the expected message. If the assertion passes when it should fail, the check is wrong.

Write two shell commands for _extras/squirrels.csv: one that counts completely blank lines (lines with no characters at all), and one that counts lines where the first field is empty (lines starting with a comma). Explain the difference between the two checks.

A typical response:

grep -c "^$" _extras/squirrels.csv
grep -c "^," _extras/squirrels.csv

The LLM explains that ^$ matches lines with no characters, while ^, catches rows where the first field is missing but later fields may have values.

A count of zero for both commands means neither problem is present. If grep -c "^$" returns a non-zero count, run grep -n "^$" _extras/squirrels.csv | head -3 to find which line numbers are blank and determine whether they represent missing records or file corruption. A non-zero count from grep -c "^," means some rows are missing their first field, which may indicate that the delimiter or encoding is not what you assumed.

A prompt like this produces a textbook comparison:

What is the difference between CSV and Parquet?

The LLM lists the standard advantages and disadvantages without connecting them to how you will actually use the data. Fix it by describing your workflow.

I have a dataset with 10 million rows that I will share with two collaborators who use Python. I will version it in Git. Most analyses filter by date and by a categorical column. Give one concrete reason to prefer CSV and one to prefer Parquet.

A typical response notes that CSV is easiest to share and inspect manually but too large for Git, while Parquet is compact and fast to filter but requires a compatible reader.

There is no easy way to check a claim like this if you don't already know the answer. A web search might be helpful, but an increasing number of "answers" are LLM-generated, and just as likely to be wrong as the one you get by prompting the LLM directly. In cases like this, you must rely on your own logic or ask a more knowledgeable colleague.