shell script, executing program with input and save input combined with output in output

Question

I am developing an assignment server, where students can upload their program solutions. It should work wither several programming languages. For testing the programs I am executing a shell script and passing the test-cases and file as arguments like that:

printf '10\n3\n+\n' | ./eval_python steps.py

Its working, that's not the problem. For the output I am getting something like that:

running python script with shell script:

$ How much steps? Step size? Counting up (+) or down (-) ? Step   0:     3
$ Step   1:     6
$ Step   2:     9
$ Step   3:    12
$ Step   4:    15
$ [..]

This format makes evaluation quite difficult and also maybe confuse students, because they don't see the inputs (when you run the python script from shell, the output looks like in the example below). For the evaluation the difficulty is, the students must not name their inputs like in this solution, it would be also okay if they just ask Steps? instead of How much steps? .

running python script directly from shell:

$ How much steps?  10
$ Step size? 3
$ Counting up (+) or down (-)? +
$ Step   0:     3
$ Step   1:     6
$ Step   2:     9
$ Step   3:    12
$ Step   4:    15
$ [..]

Is there a way to combine the inputs in the output? Or maybe other ideas how I can solve this problems?

Thanks a lot!

Can you manipulate the python code? From within there it would be easy. — Arne
– Arne, Commented Dec 15, 2017 at 11:17
@ArneRecknagel yes I could, what would be your solution? Printing the revived input? — Max
– Max, Commented Dec 15, 2017 at 11:18
First off, is the exact command to get the correct second output python steps.py "10\n" "3\n" "+\n"? — Arne
– Arne, Commented Dec 15, 2017 at 11:20
No, you are just executing python steps.py, and the scripts asks the user for input with steps = input('How many steps'). — Max
– Max, Commented Dec 15, 2017 at 11:22

Arne · Accepted Answer · 2017-12-15 14:05:53Z

1

Handling user prompts automatically is harder than it seems, and printf is unfortunately not good enough to simulate it completely. If you do not want to touch the python code itself, you'll probably have to use expect, which is a scripting language written to handle interactive programs.

If it isn't on your system (which is likely), install it with sudo apt install expect or sudo yum install expect. This crude thing here that I have mushed together should be able to handle your requirements:

expect_script.exp

#!/usr/bin/expect -f

set steps [lindex $argv 0]
set size [lindex $argv 1]
set plus_or_minus [lindex $argv 2]
set script_name [lindex $argv 3]

spawn python3 ${script_name}
# Or `spawn eval_python ${script_name}`, as long as it is a python 3 interpreter as well
# If it is python 2, all `input` calls within need to be changed to `raw_input`

expect "steps"
send -- "${steps}\r"
expect "size"
send -- "${size}\r"
expect "Counting"
send -- "${plus_or_minus}\r"
expect eof

And then call it like this:

./expect_script.exp 10 3 + steps.py

If you have questions regarding syntax or semantics, feel free to ask. But don't expect much (ha ha), I am not very good at weird Unix tools.

edit with variable amount of parameters:

expect_script.exp

#!/usr/bin/expect -f

spawn python3 [lindex $argv 0]
# Or `spawn eval_python [lindex $argv 0]`, as long as it is a python 3 interpreter as
# well. If it is python 2, all `input` calls within need to be changed to `raw_input`.

set sleep_time 0.1  # using `sleep` to wait for the next prompt is, usually,
                    # a very bad idea. Maybe you can control your input calls
                    # by including unique identifiers in them, such as 1, 2, 3..
                    # and using `except` to properly wait for them?

for {set i 1} {$i < [llength $argv]} {incr i 1} {
 sleep $sleep_time  
 send -- [lindex $argv $i]\r
}
expect eof

Call order has changed because we need to call our python script before sending the parameters its way:

./expect_script.exp test.py 10 3 +

Alternatively, change the python script to accept command line parameters. If you plan to do task automation, or reuse code, or just in general - command line parameters are almost always a better idea than user prompts. The native python ArgParse module is very good, and even if you don't want to rewrite your current code, it might still be worth it to look into it and try to apply it in the future.

edited Dec 15, 2017 at 14:05

answered Dec 15, 2017 at 12:46

Arne

20.7k11 gold badges101 silver badges107 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

Max Over a year ago

First of all thank you! We are on the right path, but if I understand this correctly, this only works for this one python script? Is there are way to make this more dynamically? For example with a different number of inputs?

Max Over a year ago

Command line parameters are of course the better way, but in this case the goal is to teach the students basic programming skills. And there will not only be functions like this one, also some small games and stuff like that. That's the reason why I need the user prompts and have to fetch them mostly automatic.

Arne Over a year ago

I added a version that can handle input of arbitrary length. Note that I use a short sleep to control that the parameters are sent to different input prompts - this is horrible style, and will break if: 1: The program does something before asking for input that takes longer than sleep_time 2: The program is run on a machine that is slower, or has hick-ups 3: You want to read input at any point in time that is not the immediate start of the program

Arne Over a year ago

If you decide to use expect and any of these cases turn up, you should rewrite the call script with expects that allow you to wait correctly for inputs. The man-pages explain in detail how the pattern matching is applied.

Gem Taylor · Accepted Answer · 2017-12-15 11:57:49Z

0

It is possible to make all your streams "demand driven", but it is not a trivial task, especially from scripting. By this I mean that each input string is only passed into the pipe when the pipe says it needs more input. Similarly, you have to collect output as soon as it is made ready. By doing this you can combine the input and output streams as they are consumed in order to make a sensible report.

Note that you will probably need to inject a few newlines to make it truly readable.

answered Dec 15, 2017 at 11:57

Gem Taylor

5,7351 gold badge11 silver badges34 bronze badges

4 Comments

Max Over a year ago

Okay. I am not really familiar with shell scripts, do you have a example how to do that?

Arne Over a year ago

I am currently trying to write one

Max Over a year ago

Thank your very much, I am also trying my best.

Gem Taylor Over a year ago

As I say, it is not trivial. I only have examples in C

Gem Taylor · Accepted Answer · 2017-12-15 15:51:41Z

Perhaps a different approach is to tell the users they must ask specific questions.

Have your harness read the output character by character (I know), and as soon as it EXACTLY matches a particular expected question, then it injects the matching answer as input, but only one time per match. Of course, once all questions have been satisfied, you can switch to a faster batched read.

You can then stream the byte-by-byte output, and the injected responses to the log file in a natural order which matches what your harness is doing. It may be good for the harness to have a time-out and print the un-consumed questions as a diagnostic aid.

This allows for question ordering, optional questions, etc that have been mentioned, but hard-wires a part of the design spec to customer requirements. Make it clear to the students that they will lose marks for not matching the prompts well enough for the automated processor, or say, for a client who provides a local language translation.

Collectives™ on Stack Overflow

shell script, executing program with input and save input combined with output in output

3 Answers 3

4 Comments

4 Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

4 Comments

4 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related