Python CGI accessing mySQL: end of script output before headers

Question

I am having trouble getting a Python CGI script to execute properly on an apache2 webserver that is running on a virtual Ubuntu 18 server. The hosting provider is DreamCompute, if that matters. I have gotten CGI scripts working under var/www/html/apps/cgi-bin/. Both helloworld.py and helloworld.pl execute fine both in the SSH terminal and in browser (Microsoft Edge).

The script giving me trouble is a Python script that accesses a MySQL database, reads data from it, then uses that data to fill some lists and generate some simple output at random (a magical spell with random effects for a tabletop RPG).

The spell generator script also executes fine in SSH, but when I try to view it in a browser it throws a 500 internal server error. The apache error logs tell me that the problem is end of script output before headers. I looked around, but couldn't find anyone with a similar problem and configuration.

EDIT: The full entry in the error log is: [Tue Apr 20 00:20:25.324101 2021] [cgid:error] [pid 17275:tid 140105176721152] [client 2607:fea8:1d41:b800:7dca:305:b11a:447f:62505] End of script output before headers: spell_generator.py, referer: http://my.website/apps/cgi-bin/

After adding and removing parts of the Python script to see how it behaves in a browser, I believe I have isolated the problem: the part of the script that connects to the MySQL database. That database is also hosted on the same Ubuntu virtual machine, and it definitely has the right kind of data in it (just strings and IDs; nothing fancy).

Here's the Python code. I've removed some documentation comments but it should be pretty straightforward:

#!/usr/bin/python3
print("Content-type: text/html\n\n");
# The above two lines are boilerplate to ensure no print statements cause 
# problems later in the program.

# Import statements
import cgi
import cgitb
cgitb.enable();
from random import choice
import mysql.connector
import sys
from enum import Enum


# Initialize enums for system comparison
class Ruleset(Enum):
    GENERIC = "GENERIC"
    GENERIC_DB = "generic_attributes"
    DND_5 = "DND5e"
    DND_5_DB = "dnd5e_attributes"
    PATHFINDER_1 = "PF1e"
    PATHFINDER_1_DB = "pathfinder1e_attributes"


# Determine what system to use, generic by default
spellSystem = Ruleset.GENERIC.value;

if (len(sys.argv)) > 1:
    if (sys.argv[1] == Ruleset.DND_5.value):
        spellSystem = Ruleset.DND_5.value;
        
    if (sys.argv[1] == Ruleset.PATHFINDER_1.value):
        spellSystem = Ruleset.PATHFINDER_1.value;
        

# === PROBLEM HERE ===

# Initialize SQL cursor, defaulting to generic
if spellSystem == Ruleset.DND_5.value:
    spellDB = mysql.connector.connect(
        host = "localhost",
        user = "RemoteUser",
        password = "password",
        database = Ruleset.DND_5_DB.value
    )
elif spellSystem == Ruleset.PATHFINDER_1.value:
    spellDB = mysql.connector.connect(
        host = "localhost",
        user = "RemoteUser",
        password = "password",
        database = Ruleset.PATHFINDER_1_DB.value
    )
else:
    spellDB = mysql.connector.connect(
        host = "localhost",
        user = "RemoteUser",
        password = "password",
        database = Ruleset.GENERIC_DB.value
    )
spellCursor = spellDB.cursor();
spellCursor.execute("SELECT ElementName FROM Element");
listHolder = spellCursor.fetchall();

# === CODE BELOW DOES NOT CAUSE PROBLEMS ===

#
# [logic that uses the SQL data]
#

# Output HTML page
print("<html> <head> <title>TEST - Magic Spell Generator</title> <link rel='stylesheet' href='../../css/style.css'> </head>");
print("<body> body contents");
print("</body>");
print("</html>");

The RemoteUser is a user account on the SQL server, which is used by "external" (non-root) programs to access the databases. password in the actual deployed script is a cryptographically secure password.

I'm not a Python expert, but the code runs with no problems when I execute it from the SSH terminal, so I don't think that bad code is to blame (although I could certainly be wrong). Here's a list of things I've tried already:

Checking for too many active SQL connections (there appears to be only one, the root user).
Making sure the script file has the correct privileges (chmod 755, same as the rest).
Making sure the necessary Python modules are installed (they are, and the Python-MySQL connector is the up to date one that works with the version of Python I'm using).
Restarting apache2.

I've spent most of today trying to find an answer. Any help or potential leads are welcome.

The apache error log should say more than just what you quoted in the question, there should be an indication of the underlying error. Without that, we can only guess. — Shadow
– Shadow, Commented Apr 20, 2021 at 1:25
@Shadow I edited the post to show the full error log entry. Have a look if you can. — Coldman
– Coldman, Commented Apr 20, 2021 at 18:26

Coldman · Accepted Answer · 2021-04-21 03:21:46Z

0

Turns out the problem wasn't actually the script trying to access the SQL database, which I figured out when I ran it properly in the SSH terminal (./spell_generator.py instead of python3 spell_generator.py). It crashed due to segmentation fault (core dumped), which implied that it was a code problem rather than a configuration problem.

After a bit of searching, I found Database connectivity through MySQL connector Python with CGI is not working, which pointed me to the real culprit: I was importing modules in the wrong order. I changed the import statements so that import mysql.connector is now the first one, and the script runs perfectly.

It shouldn't make a difference, but it does. I'm just happy.

answered Apr 21, 2021 at 3:21

Coldman

235 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Python CGI accessing mySQL: end of script output before headers

1 Answer 1

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related