How to import Apache access log into MySQL table?

Question

What is the recommended approach to import Apache access log into a MySQL table? I am thinking of a ruby/perl script + a cron job. It'll be great if there is an example or reference.

I am not sure how to handle the bookmarking of the last log entry in last import either. Suggestions are welcome. Thanks!

Import (i.e. by yourself) or write directly (i.e. by apache) ? — abatishchev
– abatishchev, Commented Aug 23, 2010 at 11:09

hpavc · Accepted Answer · 2010-08-23 04:58:15Z

3

Do you own the customlog format? Or do you have to live with whatever is there? if you own the format you can easily delimit the fields (with tabs for example) and its pretty trivial to import.

LogFormat "%{%Y-%m-%d %H:%M:%S}t\t%s\t%h\t%{ClientIP}i\t%U%q\t%{Host}i\t%B\tc1w1402@dc1app14\t%m\t%{User-Agent}i\t%{Referer}i\tDRH\t%T\t%f\t%{Via}i" ezimport

answered Aug 23, 2010 at 4:58

hpavc

1,3458 silver badges7 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

ohho Over a year ago

I can change the default LogFormat but rather not to, for compatibility reason. Is it possible for apache (on CentOS) to write 2 logs (one as default, another for MySQL import)?

hpavc Over a year ago

Yes, you can do many of them which is what the 'ezimport' indicates at the tail end.

Bhavik · Accepted Answer · 2010-08-23 11:07:23Z

1

If you can write PERL script then try out mod_perl to log directly to database... for the reference check link .

answered Aug 23, 2010 at 11:07

community wiki

Bhavik

Comments

abatishchev · Accepted Answer · 2020-01-31 01:42:54Z

0

LogFormat "yourFormat" yourLog

<VirtualHost example.com>
    ServerName example.com
    CustomLog /yourPath yourLog
</VirtualHost>

edited Jan 31, 2020 at 1:42

answered Aug 23, 2010 at 11:13

abatishchev

101k88 gold badges303 silver badges443 bronze badges

Collectives™ on Stack Overflow

How to import Apache access log into MySQL table?

3 Answers 3

2 Comments

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

2 Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related