2

I need to take a linux log file and enter the data into a mysql table, and separate each field into columns. I have been trying to use the LOAD DATA INFILE command. Only problem is, the data is not delimited and I cannot TERMINATE FIELDS BY '' because the file includes spaces in the log message. If anyone has any idea please let me know

Example of file:

2011-01-17 10:18:20 User xxxxxx requested backup of xxxxx (xxxxx)

2011-01-17 10:18:29 Stopping current full of xxxxxx, request by backuppc (backoff=)

2011-01-17 10:18:33 User xxxxxx requested backup of xxxxxx (xxxxxx )

2011-01-17 10:18:51 Backup canceled on xxxxxx (received signal=ALRM)

2011-01-17 10:18:52 Started full backup on xxxxxx (pid=xxxxxx , share=xxxxxx$)

2011-01-17 10:24:18 Started full backup on xxxxxx (pid=xxxxxx , share=xxxxxx$)

2011-01-17 11:00:01 Next wakeup is 2011-01-17 12:00:01

2011-01-17 12:00:03 Next wakeup is 2011-01-17 13:00:00

2011-01-17 13:00:01 Next wakeup is 2011-01-17 14:00:00

1 Answer 1

1

Pre-format this file. I'm assuming the blank lines in your file are not there (otherwise you can remove them using the commented code). Let's say you're using python:

parsed = open("parsed.txt", 'a')                                                                                                                                        
    with open("log.txt") as f:                                                                                                                                              
    for i, line in enumerate(f):
        # use if your lines have spaces in between
        # if i % 2 == 0:                                                                                                                                                      
        parsed.write(line.replace(" ",",",2))

Giving you a file like:

2011-01-17,10:18:20,User xxxxxx requested backup of xxxxx (xxxxx)                                                                                                                                                                                                                                                                                                                                                                   
2011-01-17,10:18:52,Started full backup on xxxxxx (pid=xxxxxx , share=xxxxxx$)                                                                                                                                                                                                                                                               
2011-01-17,10:24:18,Started full backup on xxxxxx (pid=xxxxxx , share=xxxxxx$) 
...                                                                                        

Now you can:

LOAD DATA INFILE 'parsed.txt' INTO TABLE db.my_table;
Sign up to request clarification or add additional context in comments.

1 Comment

I actually decided to go with regex to decipher my file. Was definitely the long complicated way but it will work out best in the end. When I initially started looking this is exactly what I wanted, but it turns out regex works well too. Thanks a bunch man

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.