How does it handle images?
You can make image networks (unet-like things) by chunking rectangles in 2D (with some convolution steps)... I wonder if there is an image-specific architecture a bit like this that could possibly work well?
Hierarchical Modeling (H-Nets)
21–23 of 23 posts
Re: Hierarchical Modeling (H-Nets)
#22How does it handle images?
You can make image networks (unet-like things) by chunking rectangles in 2D (with some convolution steps)... I wonder if there is an image-specific architecture a bit like this that could possibly work well?
Perhaps something like this: https://neurips.cc/virtual/2024/poster/94115 Though I haven't looked up what their actual tokenization strategy is, and whether switching to hierarchical (H-Net) chunks would be possible.
Re: Hierarchical Modeling (H-Nets)
#23> 3. H-Nets are significantly more robust to small perturbations in input data like casing, showing an avenue for creating models that are more robust and aligned with human reasoning.
If it forms a hierarchy (a tree), it seems like it wouldn't be robust to rearranging the information in a prompt.
eg if your request has a long list or a table of data, all the different permutations of that will create different trees even though they're actually the same thing.