Libvirt: Adoption of GLib library to replace GNULIB and home grown code • Hacker News Board

Re: Libvirt: Adoption of GLib library to replace GNULIB and home grown code

#21

Post by stretchcat » Sat, Jan 23, 2021, 12:49 AM UTC

inetknght wrote: post #20

lrossi wrote: post #3

Glib is OK except for the fact that it has a memory cache on top of that of malloc. This prevents tools like asan or valgrind from detecting memory related bugs. It caused my team a lot of grief, to the point that we regretted the choice of using glib in the first place. I am not sure why there is a memory cache in the first place. Malloc may have been slow in the 90s, but these days there is no reason to cache and r…

> these days there is no reason to cache and reuse allocations. No, you are wrong. Ideas like that are a big reason that apps get slower despite hardware getting faster.

Isn't the right way of doing this to use a different implementation of malloc, rather than wrapping malloc with your own meta-malloc?

Re: Libvirt: Adoption of GLib library to replace GNULIB and home grown code

#22

Post by stretchcat » Sat, Jan 23, 2021, 12:54 AM UTC

JNRowe wrote: post #7

marktangotango wrote: post #2

> These problems are common to many applications / libraries that are written in C and thus there are a number of libraries that attempt to provide a high level “standard library”. The GLib library is one such effort from the GNOME project developers that has long been appealing. I've been a fan of glib for some time, other than glib and apache libapr, what other "high level standard libraries" for C should I know ab…

I can't offer extra recommendations, but just wanted to offer full agreement. GLib is the awesome standard library I get to use everywhere thanks to gobject-introspection. From work projects in Vala, window manager¹, image viewer², video player³. It is especially useful with lua configurable projects, given the sparsity of the language itself. ¹ https://awesomewm.org/ ² https://github.com/muennich/sxiv - my user conf…

Incidentally, mpv is extensible enough to replicate sxiv's features with the right configuration and scripts. I like to think of mpv as the emacs of multimedia.

Re: Libvirt: Adoption of GLib library to replace GNULIB and home grown code

#23

Post by asveikau » Sat, Jan 23, 2021, 1:01 AM UTC

inetknght wrote: post #20

lrossi wrote: post #3

Glib is OK except for the fact that it has a memory cache on top of that of malloc. This prevents tools like asan or valgrind from detecting memory related bugs. It caused my team a lot of grief, to the point that we regretted the choice of using glib in the first place. I am not sure why there is a memory cache in the first place. Malloc may have been slow in the 90s, but these days there is no reason to cache and r…

> these days there is no reason to cache and reuse allocations. No, you are wrong. Ideas like that are a big reason that apps get slower despite hardware getting faster.

I think it becomes an application specific or domain specific thing.

For example, I've done some work with audio or video. Nobody working on that goes straight to malloc on every packet or frame. It'd just be asking for pain.

But a general purpose allocator doing its own free-list on the assumption that libc is going to suck? I think that's outdated. If you do want to support it, I think it's better to allow the caller to replace the allocator through a function pointer, rather than just do it by default in a library.

Re: Libvirt: Adoption of GLib library to replace GNULIB and home grown code

#24

Post by inetknght » Sat, Jan 23, 2021, 2:07 AM UTC

stretchcat wrote: post #21

inetknght wrote: post #20

Earlier quoted context omitted.

> these days there is no reason to cache and reuse allocations. No, you are wrong. Ideas like that are a big reason that apps get slower despite hardware getting faster.

Isn't the right way of doing this to use a different implementation of malloc, rather than wrapping malloc with your own meta-malloc?

> Isn't the right way of doing this to use a different implementation of malloc, rather than wrapping malloc with your own meta-malloc?

Yup [0].

There's even other complete libraries like tcmalloc [1] and jemalloc [2].

[0] https://stackoverflow.com/a/262481/1111557

[1] https://github.com/google/tcmalloc

[2] https://github.com/jemalloc/jemalloc

Re: Libvirt: Adoption of GLib library to replace GNULIB and home grown code

#25

Post by PaulDavisThe1st » Sat, Jan 23, 2021, 2:25 AM UTC

loeg wrote: post #4

marktangotango wrote: post #2

> These problems are common to many applications / libraries that are written in C and thus there are a number of libraries that attempt to provide a high level “standard library”. The GLib library is one such effort from the GNOME project developers that has long been appealing. I've been a fan of glib for some time, other than glib and apache libapr, what other "high level standard libraries" for C should I know ab…

If your C umbrella includes C++, Qt has a number of (non-GUI) high-level libraries that can be useful for applications.

Even the Qt folks accepted, though, that for run/event-loop integration between glib and Qt code, it was Qt that provided a way to use the glib event loop not vice versa (because Qt doesn't offer a sufficiently hook-able event loop abstraction).

Re: Libvirt: Adoption of GLib library to replace GNULIB and home grown code

#26

Post by TingPing » Sat, Jan 23, 2021, 2:37 AM UTC

asveikau wrote: post #23

inetknght wrote: post #20

Earlier quoted context omitted.

> these days there is no reason to cache and reuse allocations. No, you are wrong. Ideas like that are a big reason that apps get slower despite hardware getting faster.

I think it becomes an application specific or domain specific thing. For example, I've done some work with audio or video. Nobody working on that goes straight to malloc on every packet or frame. It'd just be asking for pain. But a general purpose allocator doing its own free-list on the assumption that libc is going to suck? I think that's outdated. If you do want to support it, I think it's better to allow the call…

glib does not always use one, the default is normal malloc and it provides another allocator api as an option.

Re: Libvirt: Adoption of GLib library to replace GNULIB and home grown code

#27

Post by asveikau » Sat, Jan 23, 2021, 3:02 AM UTC

TingPing wrote: post #26

asveikau wrote: post #23

Earlier quoted context omitted.

I think it becomes an application specific or domain specific thing. For example, I've done some work with audio or video. Nobody working on that goes straight to malloc on every packet or frame. It'd just be asking for pain. But a general purpose allocator doing its own free-list on the assumption that libc is going to suck? I think that's outdated. If you do want to support it, I think it's better to allow the call…

glib does not always use one, the default is normal malloc and it provides another allocator api as an option.

Yes, I figured that out from other posts on this thread (including one where I looked at g_malloc source).

I think libstc++ on the other hand does or did have such a cache as an always-on thing though. So not unheard of.

Re: Libvirt: Adoption of GLib library to replace GNULIB and home grown code

#28

Post by saagarjha » Sat, Jan 23, 2021, 4:28 AM UTC

inetknght wrote: post #24

stretchcat wrote: post #21

Earlier quoted context omitted.

Isn't the right way of doing this to use a different implementation of malloc, rather than wrapping malloc with your own meta-malloc?

> Isn't the right way of doing this to use a different implementation of malloc, rather than wrapping malloc with your own meta-malloc? Yup [0]. There's even other complete libraries like tcmalloc [1] and jemalloc [2]. [0] https://stackoverflow.com/a/262481/1111557 [1] https://github.com/google/tcmalloc [2] https://github.com/jemalloc/jemalloc

One reason I can think of wrapping malloc is that C is more likely to give you that than whatever OS-specific API you are going to use to get memory from the kernel (if you even have one!).

Re: Libvirt: Adoption of GLib library to replace GNULIB and home grown code

#29

Post by nn3 » Sat, Jan 23, 2021, 4:34 AM UTC

lrossi wrote: post #3

Glib is OK except for the fact that it has a memory cache on top of that of malloc. This prevents tools like asan or valgrind from detecting memory related bugs. It caused my team a lot of grief, to the point that we regretted the choice of using glib in the first place. I am not sure why there is a memory cache in the first place. Malloc may have been slow in the 90s, but these days there is no reason to cache and r…

At least for valgrind it's fairly easy to annotate custom mallocs in the source. Just someone needs to do that.

Re: Libvirt: Adoption of GLib library to replace GNULIB and home grown code

#30

Post by PaulDavisThe1st » Sat, Jan 23, 2021, 4:50 AM UTC

lrossi wrote: post #10

loeg wrote: post #4

Earlier quoted context omitted.

If your C umbrella includes C++, Qt has a number of (non-GUI) high-level libraries that can be useful for applications.

Qt is more like gtk rather than glib. Although it does come with a fantastic core library. But if you can move to C++, the standard library already offers great replacements for most glib features. Glib exists mostly because the C standard library is lacking in many aspects.

That's only partially true. You'd really something like boost to cover a lot of things that glib provides for C. glib covers many, many things that are outside of the scope of both the C++ and C standard libraries.