Congratulations, this is a great achievement in Real-Time LHC data filtering.
CERN uses ultra-compact AI models on FPGAs for real-time LHC data filtering
151–160 of 166 posts
Re: CERN uses ultra-compact AI models on FPGAs for real-time LHC data filtering
#152Earlier quoted context omitted.
Not everyone knows everything so knowledge is the new oil. I do know about linear regression even had quite some of it at university. But I still wouldn’t be able to just implement it on some data without good couple days to weeks of figuring things out and which tools to use so I don’t implement it from scratch.
Implement it...from scratch? Its literally least squares regression. Its a few lines of code. What are you trying to say here?
You have to get the data first build all data processing pipelines to get your parameters for linear regression.
Re: CERN uses ultra-compact AI models on FPGAs for real-time LHC data filtering
#153Earlier quoted context omitted.
Indeed, some examples: https://news.ycombinator.com/item?id=12340348 Neural network spotted deep inside Samsung's Galaxy S7 silicon brain (2016) https://ieeexplore.ieee.org/document/831066 Towards a high performance neural branch predictor (1999)
I'm borderline shocked that all of this extra overhead is somehow more efficient than something as simple as computing both branches or something.
The required computing resources double at every branch where you take both paths, and if you speculate ahead by 100+ instructions, with let's say up to 20 branches, it gets way out of hand.
I could see CPUs sometimes taking both paths for close, hard to predict branches. Does anyone have information on that?
Re: CERN uses ultra-compact AI models on FPGAs for real-time LHC data filtering
#154One of the authors (of one of the two models, not this particular paper) here. Just a clarification, these models are *not* burned into silicon. They are trained with brutal QAT but are put onto fpgas. For axol1tl, the weights are burned in the sense that the weights are hard-wired in the fabric (i.e., shift-add instead of conventional read-muk-add cycle), but not on the raw silicon so the chip can be reprogrammed. T…
Very cool to see you work! Early in my PhD I did some work with GNN accelerators on FPGAs (which I think later ended up in some form as a colab with some CERN or Fermilab folks) and have chatted a bit in the past with the FastML, HLS4ML, and HEP folks. I have since pivoted a lot of my PhD work (still related the HLS and EDA). But I wonder what is the current main limitation/challenges of building these trigger system…
Some CERN employees are already important contributors to KiCAD and IIRC CERN has sponsored some yosys /nextpnr development.
I would love to see more involvement from CERN in this space.
Re: CERN uses ultra-compact AI models on FPGAs for real-time LHC data filtering
#155They run at 40Mhz. This project [1] runs at 148Mhz using an open source "C/C++ to FPGA" tool to achieve realtime raytracing using integers/fixed/floating points (25Mhz with a 100% open toolchain [2]). Part of the project is currently funded by the Nlnet Foundation (the CflexHDL tool) and integration with the CERN AI tools is planned. Open source is important. DISCLAIMER: I'm author of the CflexHDL tool and the raytraced game, happy to answer questions.
[1] https://www.youtube.com/watch?v=hn3sr3VMJQU [2] https://blog.yosyshq.com/p/3d-raytracing/
Re: CERN uses ultra-compact AI models on FPGAs for real-time LHC data filtering
#156Earlier quoted context omitted.
Very cool to see you work! Early in my PhD I did some work with GNN accelerators on FPGAs (which I think later ended up in some form as a colab with some CERN or Fermilab folks) and have chatted a bit in the past with the FastML, HLS4ML, and HEP folks. I have since pivoted a lot of my PhD work (still related the HLS and EDA). But I wonder what is the current main limitation/challenges of building these trigger system…
Thank you for the comment, and the questions are great. The problems you described here are pretty much precise. In the past, and mostly now, we are replying on the commercial Vivado/Vitis HLS toolchains for the deployment of these networks through hls4ml, a template based compiler of the quantized models to the HLS projects. For this class of fully parallel (II=1) models, the tools usually give fine results, but ind…
Would you consider this open source alternative? I'm working on it https://blog.yosyshq.com/p/3d-raytracing/
Re: CERN uses ultra-compact AI models on FPGAs for real-time LHC data filtering
#157[flagged]
Re: CERN uses ultra-compact AI models on FPGAs for real-time LHC data filtering
#158Earlier quoted context omitted.
CERN doesn't build everything CERN uses: - FPAGs like this one are generally COTS. - All the experiments use GPUs which come straight from the vendors. - Most of the computing isn't even on site, it's distributed around the world in various computing centers. Yes they also overflow into cloud computing but various publicly funded datacenters tend to be cheaper (or effectively "free" because they were allocated to CER…
CERN builds almost next to nothing anymore. Half a century ago they really did do RF cavities, cooling, electronics etc. Not anymore. It is either COTS (DELL, Alterra etc.) or chiefly vendor bidding for some custom parts. Much like what NASA (from Rocketdyne, TRW to Boeing and SpaceX) or copycat ESA (Airbus, DLR, BAE's suppliers) does today. It is a project bureau. Everything is essentially outsourced, leaving a mana…
You could argue that that's what CERN should be.
Everyone needs to agree on a place to put the LHC, and a lot of the accelerator team is on sight and probably should be payed by CERN, but they have a clear set of KPIs for that: they need to get the machine up to design energy and luminosity and hold it there. The CERN accelerator and civil engineering teams are pretty impressive and have mostly done their job.
The rest of the scientific community can (and does) organize into pseudo-autonomous collaborations that draft proposals for what to do with the real-estate around the collision points and beam dumps. The vast majority of these people don't work for CERN.
Re: CERN uses ultra-compact AI models on FPGAs for real-time LHC data filtering
#159Earlier quoted context omitted.
The LHC has moved on a bit since then. Here's an open dataset that one collaboration used to train a transformer: https://opendata-qa.cern.ch/record/93940 if you can beat it with linear regression we'd be happy to know.
Thanks. The paper [1] referenced in your link follows the lagacy of the paper on the HIGGS dataset, and does not operate with quantities like accuracy and/or perplexity. HIGGS dataset paper provided area under ROC, from which one had to approximate accuracy. I used accuracy from the ADMM paper [2] to compare my results with. As I checked later, area under ROC in [1] mostly agrees with [2] SGD training results on HIGG…
For better or worse the people working on this don't really use perplexity or accuracy to evaluate models. The target is whatever you'd get for those metrics if you used the discriminants that were provided in the dataset (i.e. the GN2v01 values).
As for why accuracy and perplexity aren't reported: the experiments generally choose a threshold to consider something a "b-hadron" (basically picking a point along the ROC curve) and quantify the TPR and FPR at that point. There are reasons for this, mostly that picking a standard point lets them verify that the simulation actually reflects data. See, for example, the FPR [1] and TPR [2] "calibrations".
It's a good point, though, the physicists should probably try harder to report standard metrics that the rest of the ML community uses.
Re: CERN uses ultra-compact AI models on FPGAs for real-time LHC data filtering
#160Earlier quoted context omitted.
I'm borderline shocked that all of this extra overhead is somehow more efficient than something as simple as computing both branches or something.
The required computing resources double at every branch where you take both paths, and if you speculate ahead by 100+ instructions, with let's say up to 20 branches, it gets way out of hand. I could see CPUs sometimes taking both paths for close, hard to predict branches. Does anyone have information on that?
I believe the way things are currently trending is that architectures might turn some short hard to predict branches into predicated instructions instead (similar to x86 CMOV or some ARM conditional execution instructions). Outside of short branches the overhead for loading up to 2 instructions for every 1 that gets executed can be too costly. Branch predication on SIMD/SIMT instructions is already the way things work for GPUs and AVX256/512 from my understanding.