Live data from Hacker News

Proxmox VE: Import Wizard for Migrating VMware ESXi VMs

forum.proxmox.com

111–118 of 118 posts

Re: Proxmox VE: Import Wizard for Migrating VMware ESXi VMs

#111
post #69

Earlier quoted context omitted.

Unprivileged LXCs? Interesting, I thought containers would require privileged LXC. At least, that it my takeaway from trying to run Podman in a nesting enabled, but unprivileged LXC under non-root user. I kept running into > newuidmap: write to uid_map failed: Operation not permitted I tried googling it, tried some of the solutions, but reached the conclusion that it's happening because the LXC is not privileged.

I once wrote a post about Docker in unprivileged LXC on ZFS [1]. The post is a little bit outdated, as it is much simpler today with ZFS 2.2.0, which is natively supported. There's also a more recent post that shows how to run rootless docker [2], with updated uid-mappings. Both may be helpful, have a look. The advantage of using LXC for me is resource consumption and separation of concerns. I have about 35 Docker co…

Thank you, this made me realize I assigned a wrong (too little) number for uids. It did not fix my issue however. I still see

  (dev) $ podman info
  ERRO[0000] running `/usr/bin/newuidmap 3427 0 1000 1 1 100000 65536`: newuidmap: open of uid_map failed: Permission denied
  Error: cannot set up namespace using "/usr/bin/newuidmap": exit status 1
I tried a solution I found on Red Hat's Customer Portal:

  (root) # setcap cap_setuid+ep /usr/bin/newuidmap
  (root) # setcap cap_setgid+ep /usr/bin/newgidmap
Also did not work. I can run

  (root) # podman info 
just fine as root. This leads me to believe there are some other problems with my non-root user permissions.

EDIT: It probably makes little sense, to run rootless on top of an already unprivileged LXC. I just wanted to give vscode server it's own non-root user in there. Oh well...

Re: Proxmox VE: Import Wizard for Migrating VMware ESXi VMs

#112
post #111

Earlier quoted context omitted.

I once wrote a post about Docker in unprivileged LXC on ZFS [1]. The post is a little bit outdated, as it is much simpler today with ZFS 2.2.0, which is natively supported. There's also a more recent post that shows how to run rootless docker [2], with updated uid-mappings. Both may be helpful, have a look. The advantage of using LXC for me is resource consumption and separation of concerns. I have about 35 Docker co…

Thank you, this made me realize I assigned a wrong (too little) number for uids. It did not fix my issue however. I still see (dev) $ podman info ERRO[0000] running `/usr/bin/newuidmap 3427 0 1000 1 1 100000 65536`: newuidmap: open of uid_map failed: Permission denied Error: cannot set up namespace using "/usr/bin/newuidmap": exit status 1 I tried a solution I found on Red Hat's Customer Portal: (root) # setcap cap_s…

Yes, just start from scratch and provide uid-mappings from the beginning. Looks like those uids were set from before adding the mappings and it is trying to access uids it is not allowed to access.

I used rootless docker in rootless lxc because the Postgres Docker (e.g.) will try to setup a non-root user by default. In a rootless LXC, this means it will try to access very large uids (>100000), which are not available, unless explicitly prepared.

Re: Proxmox VE: Import Wizard for Migrating VMware ESXi VMs

#113
post #111

Earlier quoted context omitted.

Thank you, this made me realize I assigned a wrong (too little) number for uids. It did not fix my issue however. I still see (dev) $ podman info ERRO[0000] running `/usr/bin/newuidmap 3427 0 1000 1 1 100000 65536`: newuidmap: open of uid_map failed: Permission denied Error: cannot set up namespace using "/usr/bin/newuidmap": exit status 1 I tried a solution I found on Red Hat's Customer Portal: (root) # setcap cap_s…

Yes, just start from scratch and provide uid-mappings from the beginning. Looks like those uids were set from before adding the mappings and it is trying to access uids it is not allowed to access. I used rootless docker in rootless lxc because the Postgres Docker (e.g.) will try to setup a non-root user by default. In a rootless LXC, this means it will try to access very large uids (>100000), which are not available…

That actually did not do anything different for me. I did the following:

1) Created a new LXC.

2) As root, I created a new user "devel"

3) For the "devel" user set both subuid and subgid to devel:100000:65536

4) As root, installed podman

5) In another SSH session, logged in as "devel" and ran "podman version"

Same error as before. This is in a Debian 12 LXC running on Proxmox.

Re: Proxmox VE: Import Wizard for Migrating VMware ESXi VMs

#114
post #113

Earlier quoted context omitted.

Yes, just start from scratch and provide uid-mappings from the beginning. Looks like those uids were set from before adding the mappings and it is trying to access uids it is not allowed to access. I used rootless docker in rootless lxc because the Postgres Docker (e.g.) will try to setup a non-root user by default. In a rootless LXC, this means it will try to access very large uids (>100000), which are not available…

That actually did not do anything different for me. I did the following: 1) Created a new LXC. 2) As root, I created a new user "devel" 3) For the "devel" user set both subuid and subgid to devel:100000:65536 4) As root, installed podman 5) In another SSH session, logged in as "devel" and ran "podman version" Same error as before. This is in a Debian 12 LXC running on Proxmox.

I am also using Docker in Debian 12 LXC on Proxmox. I am not sure what has gone wrong here.

Re: Proxmox VE: Import Wizard for Migrating VMware ESXi VMs

#115
post #113

Earlier quoted context omitted.

That actually did not do anything different for me. I did the following: 1) Created a new LXC. 2) As root, I created a new user "devel" 3) For the "devel" user set both subuid and subgid to devel:100000:65536 4) As root, installed podman 5) In another SSH session, logged in as "devel" and ran "podman version" Same error as before. This is in a Debian 12 LXC running on Proxmox.

I am also using Docker in Debian 12 LXC on Proxmox. I am not sure what has gone wrong here.

Was there anything extra you have done on the Host itself?

Re: Proxmox VE: Import Wizard for Migrating VMware ESXi VMs

#116
post #115

Earlier quoted context omitted.

I am also using Docker in Debian 12 LXC on Proxmox. I am not sure what has gone wrong here.

Was there anything extra you have done on the Host itself?

I described the full process here [1]. The only thing that seems to differ is podman for you.

Ah, I see:

> 4) As root, installed podman

I installed docker as the non-root user. See my Mastodon post, there's a specific procedure to install Docker in a user namespace ("devel" in your case).

[1]: https://du.nkel.dev/blog/2021-03-25_proxmox_docker/

Re: Proxmox VE: Import Wizard for Migrating VMware ESXi VMs

#117
post #69

I use Proxmox since 5 years at home. Mostly docker nested in unprivileged LXCs on ZFS for performance reasons. I love the reliability. Proxmox has never lost me. They churn out constant progress without making too much noise. No buzzwords, no advertising, just a good reliable product that keeps getting better.

Unprivileged LXCs? Interesting, I thought containers would require privileged LXC. At least, that it my takeaway from trying to run Podman in a nesting enabled, but unprivileged LXC under non-root user. I kept running into > newuidmap: write to uid_map failed: Operation not permitted I tried googling it, tried some of the solutions, but reached the conclusion that it's happening because the LXC is not privileged.

Replying to my own comment for posterity. I was able to figure it out!

In the LXC, I created a new non-root user and then set it's uid and git to a LOWER number than what every tutorial about rootless Podman recommends (devel being my non-root user here):

  # cat /etc/subuid /etc/subgid
  devel:10000:5000
  devel:10000:5000
Then I also had to edit configuration for the LXC itself on the Proxmox host to allow tun and have it created on container boot:

   # cat /etc/pve/lxc/{lxc_vmid}.conf
   ...truncated...
   lxc.cgroup2.devices.allow: c 10:200 rwm
   lxc.mount.entry: /dev/net dev/net none bind,create=dir
Note: I have no idea why a lower number of ids works...

Re: Proxmox VE: Import Wizard for Migrating VMware ESXi VMs

#118
post #117
post #69

Earlier quoted context omitted.

Unprivileged LXCs? Interesting, I thought containers would require privileged LXC. At least, that it my takeaway from trying to run Podman in a nesting enabled, but unprivileged LXC under non-root user. I kept running into > newuidmap: write to uid_map failed: Operation not permitted I tried googling it, tried some of the solutions, but reached the conclusion that it's happening because the LXC is not privileged.

Replying to my own comment for posterity. I was able to figure it out! In the LXC, I created a new non-root user and then set it's uid and git to a LOWER number than what every tutorial about rootless Podman recommends (devel being my non-root user here): # cat /etc/subuid /etc/subgid devel:10000:5000 devel:10000:5000 Then I also had to edit configuration for the LXC itself on the Proxmox host to allow tun and have i…

Ah.. apologies for my misguidance below. It made me realize that I wrote these blog posts with a VM on another Hypervisor, not on my Proxmox/LXC (just the Docker guide that I haven't yet transitioned to rootless-in-unprivileged-lxc).

See the explanation here [1]: Unprivileged LXC on Proxmox seem to be restricted to the uid range below 65536 UIDs/GIDs (to be used _inside_ the LXC -> to be mapped to > 100000:165536 outside the LXC/on the host).

In order to use subuids/gids > 65536 inside the LXC, add a mapping to the LXC config:

    root:100000:65536
to /etc/subgid and /etc/subuid.

Now you'll have 100000 to 165536 available inside the LXC, where you can add:

    devel:100000:65536
to the /etc/subgid and /etc/subuid inside the LXC, for nested rootless podman/docker.

As a consequence, you're mapping the devel user to the same range as the LXC root user. In other words, processes inside the LXC and inside the rootless podman could run in the same namespace on the Proxmox host. If you don't want that, you'll need to provide additional ranges to the LXC (e.g. root:100000:165536 and then map `devel` to (e.g.) 200000 to 265536 (devel:200000:265536).

* I did not actually test all stated above.

[1] https://forum.proxmox.com/threads/how-to-use-uids-gids-highe...

Post reply on HN