Python syntax highlighter for Python

Question

Is there a simple (possibly with no dependencies) syntax highlighter witch tokenizes Python source code? I need it to return the token type, the offset from the beginning of the code and token length.

I wanted to use Python build-in tokenize but it does not return the offsets.

I also looked to a few of standard syntax highlighters but they all tokenize other languages as well, which is overkill for my case. All I'm looking for is simple (and fast) Python syntax highlighter.

Thanks in advance!

Noufal Ibrahim · Accepted Answer · 2012-04-29 18:02:42Z

7

I don't know about the offset but you can get back a stream of tokens from a Python snippet using pygments. It's a single standalone Python library as far as I know. The stream of tokens can then be used to do whatever you want. I have an example that renders a snippet of Python onto a pygame surface with syntax highlighting using Pygments. It tokenises the input line by line.

answered Apr 29, 2012 at 18:02

Noufal Ibrahim

73.2k13 gold badges140 silver badges174 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

alexzander Over a year ago

pygments is slow, very slow, not worth if you want performance

alexzander Over a year ago

use pysyntect for fast computation, the library is made in rust, but can be accesible from python code

John Erck · Accepted Answer · 2012-04-29 17:59:10Z

2

Checkout http://oneqonea.blogspot.com/2012/04/how-do-i-add-syntax-highlighting-to-my.html

It's a really easy "SyntaxHighlighter for Blogger" tutorial with screenshots and everything.

You should be up and running in only a few minutes.

In your case since you're looking for a "simple (and fast)" syntax highlighter without any unnecessary overhead, on your implementation simply include only the shBrushPython.js library file like this:

<head>
    <script src='http://alexgorbatchev.com/pub/sh/current/scripts/shBrushPython.js' type='text/javascript'/>
</head>

Hope this helps. Happy coding.

answered Apr 29, 2012 at 17:59

John Erck

9,5388 gold badges64 silver badges71 bronze badges

1 Comment

Ecir Hana Over a year ago

Thanks, but I wanted it to be in Python.

Collectives™ on Stack Overflow

Python syntax highlighter for Python

2 Answers 2

2 Comments

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

2 Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related