Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Understanding and writing a JPEG decoder in Python

yasoob.me

Post reply on HN
Print view View original

31–40 of 48 posts

Re: Understanding and writing a JPEG decoder in Python

#31

Post by eru »

userbinator wrote: post #10
enriquto wrote: post #8

Earlier quoted context omitted.

> Even if you do write code, it is in C/C++ and not accessible to a wide group of people. To be honest, writing a jpeg decoder in C seems easier and more natural than doing it in python.

As someone who has done it in C, I would both agree and disagree --- reading the file format will definitely be easier since C lets you pick bits/bytes/words/etc. off the stream directly, but on the other hand the high-level structures (looping, etc.) would probably be easier with Python. Overall, seeing as OP's Python implementation is less than 300 LoC while my C implementation was closer to 750, the Python might b…

The main benefit I see here of Python over C is when your program doesn't work (yet): Python has nicer error handling built straight into the language.

If you just want to present a working implementation, both C and Python can express readable versions.

Re: Understanding and writing a JPEG decoder in Python

#32

Post by eru »

enriquto wrote: post #26
smabie wrote: post #23

Earlier quoted context omitted.

Using Haskell and a parser combinator would probably be the easiest

Easier for whom? Most programmers would not be able to follow that code. On the other hand, a really simple C or Python algorithm is more or less universally readable.

It's not so much about the algorithm, but the parsing itself.

Parser combinators give you one of the most straightforward ways to express parsers.

You are right, that the language you express your parser combinators in vs the language you express an alternative solution in also will have an impact.

You can in theory express parser combinators in Python. But it's gonna be a bit ugly, because Python has relatively clunky syntax for functions.

Some people have tried writer parser combinator libraries for eg JavaScript. See https://github.com/GregRos/parjs

I just found a parser combinator library for Python https://pypi.org/project/parsita/ But it looks like they have to abuse some Python magic to make the code readable.

Re: Understanding and writing a JPEG decoder in Python

#33

Post by enriquto »

eru wrote: post #32
enriquto wrote: post #26

Earlier quoted context omitted.

Easier for whom? Most programmers would not be able to follow that code. On the other hand, a really simple C or Python algorithm is more or less universally readable.

It's not so much about the algorithm, but the parsing itself. Parser combinators give you one of the most straightforward ways to express parsers. You are right, that the language you express your parser combinators in vs the language you express an alternative solution in also will have an impact. You can in theory express parser combinators in Python. But it's gonna be a bit ugly, because Python has relatively clun…

Alright, but jpeg decoding is nearly all algorithm, there's almost no parsing. Do you have a particular haskell jpeg decoder in mind that uses parser combinators?

Re: Understanding and writing a JPEG decoder in Python

#34

Post by eru »

enriquto wrote: post #33
eru wrote: post #32

Earlier quoted context omitted.

It's not so much about the algorithm, but the parsing itself. Parser combinators give you one of the most straightforward ways to express parsers. You are right, that the language you express your parser combinators in vs the language you express an alternative solution in also will have an impact. You can in theory express parser combinators in Python. But it's gonna be a bit ugly, because Python has relatively clun…

Alright, but jpeg decoding is nearly all algorithm, there's almost no parsing. Do you have a particular haskell jpeg decoder in mind that uses parser combinators?

No. But the author of the original comment mentioning Haskell https://news.ycombinator.com/item?id=23841865 might have had?

Re: Understanding and writing a JPEG decoder in Python

#35

Post by mehh »

yasoob wrote: post #2

Hi everyone! OP here. Why write another article on JPEG when there are already hundreds of articles on the internet? Well, normally when you read articles on JPEG, the author just gives you details about what the format looks like. You don’t implement any code to do the actual decompression and decoding. Even if you do write code, it is in C/C++ and not accessible to a wide group of people. I tried to change that thr…

Very cool!

It's not that far off going for an MPEG-2 video decoder next, highly recommend (a long time since I did, so take with a pinch of salt). That would be great to see in python to make that realm more accessible.

Re: Understanding and writing a JPEG decoder in Python

#36

Post by Ahmd72 »

Thanks for making this, a couple of months ago I was working on a jpeg project and needed to understand how it works. I was searching wildly for a tutorial with a python implementation, unfortunately never found one. I found this one in C++ though, it goes a lot more in depth https://www.youtube.com/playlist?list=PLpsTn9TA_Q8VMDyOPrDKm...

Re: Understanding and writing a JPEG decoder in Python

#37

Post by kerkeslager »

smabie wrote: post #25
kerkeslager wrote: post #21

Earlier quoted context omitted.

So, what elements of Python do you consider boilerplate? I say this not as someone defending Python, but as someone writing a programming language.

Lack of function currying and anonymous function arguments. Also having to explicitly convert a zip, map, etc to a list. Also the lambda keyword.

Hmm, I think you're using the word "boilerplate" differently than it is usually used. The changes you're proposing are functional changes that can't be transparently applied by a compiler/interpreter without changing behavior.

Currying is a subset of partial application, which Python supports via the standard library. Universal currying would be terser in some situations, but it conflicts with Python's args/kwargs implementation. Frankly, I don't think currying is a good thing even in languages such as Haskell where it doesn't conflict, as it makes it much less clear what the flow is when functions can be called with different signatures.

Anonymous function arguments, I'm just not sure what you're talking about there.

Explicitly converting higher order function results to lists is not boilerplate, it's a meaningful difference. The explicit conversion allows you to determine when code is actually executed, instead of having it executed eagerly. Universal lazy application like in Haskell has broader implications which are generally negative, particularly with regards to performance and being able to reason about execution: production Haskell often turns this off, and for good reason.

The lambda keyword is arguably boilerplate, but I'm skeptical of whatever alternate syntax you're proposing there. C#-style anonymous functions wouldn't work with Python syntax. If you're proposing people use the λ symbol, I'm going to frankly say that's a bad idea, because most developers' keyboards don't have that symbol. Like it or not, English with a latin keyboard is the lingua franca of software--if you want to target another language that would be reasonable, but that's a big choice with broader implications.

Re: Understanding and writing a JPEG decoder in Python

#38

Post by kerkeslager »

pansa2 wrote: post #30
kerkeslager wrote: post #21

Earlier quoted context omitted.

So, what elements of Python do you consider boilerplate? I say this not as someone defending Python, but as someone writing a programming language.

It’s not so much that Python requires boilerplate - all executable code does. For example, the article includes code to read the input data from a file, which is not relevant to the algorithm and could be omitted from a pseudocode version. Similarly, executable code requires unnecessary detail, like the article’s `Stream` class. In pseudocode this could be replaced by simply saying “get the next N bits”.

I don't think we're defining "boilerplate" the same way, then. Reading from a file isn't something that every executable does, so when you add code for it, it's because you want your program to do it. Boilerplate is things that have to be added to every program regardless of whether they communicate anything that differentiates your program: main(), for example, or class in Java when you're not actually trying to define a class that will be instantiated by later code.

Re: Understanding and writing a JPEG decoder in Python

#39

Post by smabie »

kerkeslager wrote: post #37
smabie wrote: post #25

Earlier quoted context omitted.

Lack of function currying and anonymous function arguments. Also having to explicitly convert a zip, map, etc to a list. Also the lambda keyword.

Hmm, I think you're using the word "boilerplate" differently than it is usually used. The changes you're proposing are functional changes that can't be transparently applied by a compiler/interpreter without changing behavior. Currying is a subset of partial application, which Python supports via the standard library. Universal currying would be terser in some situations, but it conflicts with Python's args/kwargs im…

How about some variation of: (x, y) -> x*y for lambdas?

Scala's method for dealing with laziness through views and iterators works really well.

Re: Understanding and writing a JPEG decoder in Python

#40

Post by cgrealy »

smcl wrote: post #29
cgrealy wrote: post #28

Earlier quoted context omitted.

> Even if you do write code, it is in C/C++ and not accessible to a wide group of people. Eh?? There are literally millions of C++ programmers and high quality C++ compilers are available on almost every platform for free. How is that “not accessible”?

Equally there are millions who don't use C or C++. I think it’s pretty fair to describe this as a "wide group of people" who would consider C or C++ inaccessible.

That’s a nonsense argument.

For any given programming language, the number of people not using it outweighs the number of people using it by several orders of magnitude.

It’s hard to get accurate numbers, but a quick google estimates there is roughly double the number of Python programmers than C++ programmers. I find it very difficult to imagine there isn’t some overlap there and even more difficult to imagine a competent Python programmer looking at C++ code and being completely unable to read it.

Post reply on HN