I'm great at broad strategies with no detail!
Run an Apache server. Put up a web-interface(1). User log-ins and all that jazz. The students navigate to the assignment they're working on, paste their code in a field(2), and submit it. Everything(3) gets stored in a SQL database, and kicks off the associated unit-testing script(4). Good unit-tests have descriptive reports about why units fail and test every unit independently, ie what you tell the student about what didn't work. These results are likewise stored in the database and reported to the student.
But maybe a little detail:
(1) You could have them directly ftp their data to a permissions restricted folder, or let them access the database directly, but you want to give them feedback about how they did. Plus kids these days are used to web-interfaces. And it's better to secure a website and validate input fields rather than exposing a database. Take that as a big hint that you will have students that will try to compromise your system.
(2) or select a file or whatever. Work with me here. You could also give them a space to ask questions, give excuses, rant, and/or beg forgiveness.
(3) Everything everything. What was submitted, by who, for what, from where, with a time-stamp. The lesson itself too: the problem, the example code, the unit testing, the scripts it uses, everything. You want to be able to add lessons to this and you want to be able to know what the heck happened when a student breaks it.
(4) The code the students submit, either a block of code, a function, or a whole program is loaded into the unit-testing framework which takes the code and runs a set of unit-tests against it. These are tests that you write. Each test sees if the code breaks in a specific way.
Other things to consider when making an automated system for programming homework:
- Brand new programmers need hand-holding. They won't know they need to #include to use printf(). Give them the lines above and below the lesson section. Don't even let them edit those lines. Give them one blank box to fill in.
- The lesson plan should include debugging other people's code. Including simple syntax errors and fundamental design issues.
- Reading gcc error messages is a skill
- Eventually students need the experience of being given a blank slate and making a whole program, with multiple files, that compiles on different architectures, in Malbolge.
- Do not presume how the students will solve the problem. I've had classes where half the class initially failed an assignment because the automated test assumed "0" was the "front" of the array.
- Ideally, problems should be adjusted on a student by student basis, to keep people from just swapping code. If it were a physics class, it'd be as simple as generating a set of slightly different variables for each student so everyone has to come up with their own solution. For a programming class, it will probably only be applicable for some lessons.
- The linux guy will submit code that doesn't contain the standard windows end-line combo. You can't fail him because of that.
- You're taking code written by others and executing it. SANDBOX IT. Likewise, sanitize the user input to the webserver. Always beware little Bobby Tables.
- Writing the unit tests for the lesson plan which covers unit-testing is going to be very meta. Take some asprin. Good luck.