-->

DEVOPSZONES

  • Recent blogs

    How to get unique value from a column in a csv file

     You can extract the unique values from a CSV column in several ways depending on the tools you prefer. Here are the most common methods:


    🐍 1. Python (pandas)

    If you have Python installed, this is the easiest:

    import pandas as pd
    
    # Read CSV file
    df = pd.read_csv("yourfile.csv")
    
    # Replace 'ColumnName' with the exact column header
    unique_values = df["ColumnName"].unique()
    
    # Print as a list
    print(unique_values)
    

    Save to another file:

    pd.Series(unique_values).to_csv("unique_values.csv", index=False)
    

    🖥️ 2. Command line (Linux / macOS)

    If the CSV is simple (no embedded commas/quotes):

    cut -d',' -f2 yourfile.csv | sort -u
    
    • -d',' → delimiter is comma

    • -f2 → second column (change to your column number)

    • sort -u → sort and remove duplicates.

    ⚠️ If your CSV has quoted fields with commas inside, use a CSV-aware tool like csvkit:

    csvcut -c "ColumnName" yourfile.csv | tail -n +2 | sort -u
    

    📊 3. Excel / Google Sheets

    1. Open the CSV in Excel/Sheets.

    2. Select the column.

    3. Go to Data → Remove Duplicates (Excel) or Data → Data cleanup → Remove duplicates (Google Sheets).

    4. The remaining values are the unique ones.


    ✅ Tip

    • Always check the exact column header (case-sensitive) when using pandas or csvcut.

    • For very large files, command-line tools (csvkit, awk) are usually fastest.

    Choose the method that fits the tools you already use.

    No comments