Projectswc
A command-line program that counts the bytes, lines, words and characters in a file, or in whatever is piped into it, and matches the Unix wc tool. It is small to write, but getting every count right forces you to understand how text is really stored and how a program reads input it can't hold all at once.
From Coding Challenges #1 ↗ by John Crickett
No sign-up. You tell the mentor your language: Java, Python or Node.js.
What you'll learn · 6 concepts
In the order the steps use them. Each one links to the best places to learn it, and you explain it in your own words once its steps are built.
wc counts bytes with one option and characters with another, and the sample file contains non-ASCII text, so the two differ (342190 and 339292). The character count depends on the locale, and has to count code points the way wc does.
Covers: code points and UTF-8 · code units and surrogate pairs · grapheme clusters · the locale · invalid bytes
The Absolute Minimum Every Software Developer Must Know About Unicode in 2023 (Nikita Prokopov) ↗ · Unicode FAQ: UTF-8, UTF-16, UTF-32 and BOM ↗ · GNU coreutils manual: wc invocation ↗
Every count must work on files of any size and on piped input, without loading everything.
Covers: short reads and end of input · one pass for all the counts · state across chunk boundaries · what goes wrong if you load everything
The word count is this machine, and its state has to survive from one chunk to the next.
Covers: the two states and their transitions · what counts as whitespace
Finite-state machine (Wikipedia) ↗ · Whitespace character (Wikipedia) ↗
The line count is a newline count, and the line endings in the file affect the byte count.
Covers: counting newlines, not lines · line feed versus carriage return and line feed
Newline (Wikipedia) ↗ · POSIX base definitions: Definitions (see Line) ↗
The last step reads standard input when no file is named, and a missing file should give an error on standard error and a non-zero exit code.
Covers: standard input, output and error · exit codes
stdin(3), Linux manual page ↗ · Bash manual: Exit Status ↗ · The Art of Unix Programming: Basics of the Unix Philosophy ↗
Every check is "matches wc", and the character count only matches under the same locale.
Covers: pinning the reference's version and locale · choosing nasty inputs
What you'll build · 7 steps
Steps of 30 to 60 minutes, in your own editor and language. Each one has a check, so you know when it works.
Create the project, download the challenge's sample text file, and make sure the real wc is available to compare against.
Works whenThe real wc reports 342190 bytes, 7145 lines and 58164 words for the sample file.
Concepts: testing against a reference implementation
With -c and a file name, print the number of bytes in the file, followed by the file name.
Works whenThe byte count for the sample file is 342190, the same as wc.
Concepts: text encoding · streaming in constant memory
With -l, print the number of lines in the file.
Works whenThe line count for the sample file is 7145, the same as wc.
Concepts: what a line is, and line endings
With -w, print the number of words in the file.
Works whenThe word count for the sample file is 58164, the same as wc.
Concepts: counting words with a state machine · streaming in constant memory
With -m, print the number of characters in the file, according to the current locale.
Works whenUnder a UTF-8 locale, the character count for the sample file is 339292, the same as wc under the same locale.
Concepts: text encoding · streaming in constant memory · testing against a reference implementation
With no option, print lines, words and bytes together (the same as -l, -w and -c), in that order, followed by the file name.
Works whenThe sample file gives 7145, 58164 and 342190, laid out like wc's output.
Concepts: streaming in constant memory
When no file name is given, read from standard input instead.
Works whenPiping the sample file into the program with -l gives 7145.
Concepts: the command-line contract · streaming in constant memory