FreeStatisticalTools.com
Lesson 2 of 11 · 20 min

Clean and code your survey data

Find and fix duplicates, missing-value codes, inconsistent spelling, impossible values and reverse-worded items, then compute scale scores.

Raw survey exports are never ready for analysis. Online forms create duplicate submissions, people type “female”, “F” or “Female ” for the same answer, old datasets use −99 for missing values, and some questionnaire items are worded in the opposite direction from the rest. If you analyse the raw file, −99 is treated as a real number of hours and “F” becomes a third gender. This lesson walks through cleaning the course survey step by step, in the order that avoids problems.

Try it in the tool

Clean the raw survey export

  1. Download student-survey-raw.csv and drop it on the data cleaning tool (or use Upload file).
  2. Open Health check. It lists 20 things to check.
  3. Apply the fixes described in this lesson, in order, and watch the cleaning log grow.
  4. Finish with Export & analyse: download a clean CSV and the cleaning log.

Step 1: look before you change anything

Open the Data view and scroll through the rows. You already see problems: student S018 has a SleepHours value of −99, S013 is recorded as “F”, S019 as “female”, and S041 is 199 years old. Then open the Variable view. Check that each variable has the right type: ID, Gender, Faculty, Counselling and Passed are text; everything else is numeric. Add variable labels such as “ST1: I feel overwhelmed by my coursework” so the codebook makes sense months later.

Step 2: run the health check

The health check finds 20 issues in the raw file. The important ones are:

  • 3 exact duplicate rows (the same student submitted the form twice). Remove them: the file goes from 243 to 240 rows.
  • −99 codes in StudyHours, SleepHours, ST4, SS2 and Midterm. Mark −99 as a missing-value code so that these cells become blank and are excluded from calculations, instead of dragging means down.
  • Gender spelling: extra spaces and “Male”/“male”, “Female”/“female”. Trim spaces and apply Title Case.
  • Extreme values in Age, StudyHours and other variables. Do not delete these automatically; review them one by one in step 4.
  • Straight-lining: respondents who gave the same answer to all four stress or all four support items. With only four items per scale, identical answers are often genuine, so review rather than delete. Remove a respondent only when there is other evidence of careless answering, such as an implausibly fast completion time or failed attention checks.

Step 3: fix what the automatic checks miss

After Title Case, Gender still has four categories, because “F” and “M” are not spelling variants of “Female” and “Male”. Use Recode values on Gender, into the same variable, with the rules "F" = "Female" and "M" = "Male". Check the frequency table afterwards: there should be exactly two categories, 134 female and 106 male.

Step 4: impossible and extreme values

Distinguish between three cases. An impossible value is a data-entry error: an age of 199 or of 2 for a university student. The health check flagged 199 as extreme, but 2 is not extreme in z-score terms and slips through, which is why you should also check the minimum and maximum of every variable against its possible range. Set impossible values to missing (Recode: lo thru 16 = missing and 100 thru hi = missing on Age), or correct them from the original paper form if you have it. A genuine extreme value, such as a student who studies 53 hours a week, is real data; keep it and deal with it in the analysis (Lesson 5). A missing-value code, like −99, was handled in step 2.

Step 5: reverse-code negatively worded items

Three of the stress items are worded so that agreeing means more stress (“I feel overwhelmed by my coursework”). ST3 is worded the other way: “I feel in control of my studies”. A student who strongly agrees with ST3 is less stressed, so before combining the items, ST3 must be reversed. Use Reverse-code items, select ST3, and set the scale minimum to 1 and maximum to 5. New values are min + max − old, so 1 becomes 5, 2 becomes 4 and 3 stays 3. Save into a new variable, ST3_r, so the original remains available. In Lesson 4 you will see what happens to reliability if you forget this step.

Step 6: compute the scale scores

Use Compute variable to create the scale means: Stress = mean(ST1, ST2, ST3_r, ST4) and Support = mean(SS1 to SS4). The mean() function skips missing items, so a student who skipped one stress item still gets a score from the other three. If you prefer, require a minimum number of answered items (for example, 3 of 4). Means are easier to interpret than sums because they stay on the original 1–5 scale.

Step 7: export and document

From Export & analyse, download the clean CSV (or an Excel file with data, codebook and cleaning log in three sheets) and keep the raw file untouched. Never overwrite raw data: if you make a mistake, you can always repeat the cleaning. The cleaning log lists every step, which is exactly what you need for the data screening paragraph of your methods chapter.

From the next lesson on, every exercise opens the tools with student-survey-clean.csv already loaded, so everyone works with identical numbers. It is the result of the steps above: 240 students, the two impossible ages set to missing, and the new variables ST3_r, Stress and Support. If your own cleaning gives slightly different numbers, compare your log with the steps above.

Reporting your data screening

Data were screened before analysis. Three duplicate submissions were removed, leaving 240 participants. Missing-value codes (−99) were recoded as missing, and two impossible age values were set to missing. In total, seven values (fewer than 0.2% of responses) were missing; they were handled by pairwise or listwise deletion as appropriate. Item ST3 was reverse-coded, and scale scores were computed as the mean of the items.

Key takeaways

  • Keep the raw file; clean a copy and keep a log.
  • Order matters: duplicates and missing codes first, then spelling, then impossible values, then reverse-coding, then scale scores.
  • Automatic checks miss things: always look at frequency tables and minimum and maximum values.
  • Delete cases only with a stated rule, and report how many were removed and why.

Check your understanding

Answer each question to see the explanation. Finishing the quiz marks this lesson as complete.

Q1Why must −99 be defined as a missing-value code before analysis?
Q2A five-point item is reverse-coded with new = min + max − old. What does a 2 become?
Q3A student reports studying 53 hours a week. The value is unusual but possible. What should you do during cleaning?
Q4After applying Title Case, Gender still has the categories Female, Male, F and M. What is the right fix?