Could someone explain the difference between this and Avro or Parquet? Do they serve the same purpose?
I don't know as much about the internals of Avro, but I know it is a bit different from Parquet, in that it can be used to serialize and deserialize smaller amounts of data. It is used to store large datasets in files, although it will in most cases be less space efficient than Parquet. It has also been used as a way of embedding complex structures into other systems (similarly to how JSON can be embedded in a database), or for serializing individual structures between systems. The binary representation of Avro needs to be read into a system-specific format like a C/C++ struct/object, Java object, etc. for consumption.
In contrast, Arrow is designed to represent a list of objects/records efficiently. It is designed to allow for a chunk of memory to handed to a lightweight language-specific container that can immediately reference into the memory to grab a specific value, without reading each of the records into it's own individual object or structure.