I'm wondering how they're determining a correct answer. I know, for sure, that one of my answers was correct but it was marked incorrect. I'm wondering if I need to include specific keywords in my answer? How detailed do I need to be?
SREBench Competition
11–20 of 28 posts
Re: SREBench Competition
#12I'm wondering how they're determining a correct answer. I know, for sure, that one of my answers was correct but it was marked incorrect. I'm wondering if I need to include specific keywords in my answer? How detailed do I need to be?
try to be as detailed as possible. With text sometimes it's hard to determine how close your answer was to the correct one.
EDIT: After asking this instead of responding back to me they just removed it. But it obviously shows how lacking their system is.
Re: SREBench Competition
#13Earlier quoted context omitted.
try to be as detailed as possible. With text sometimes it's hard to determine how close your answer was to the correct one.
How did a user with 3 strawberries as their name achieve a 3333333.33% success rate? EDIT: After asking this instead of responding back to me they just removed it. But it obviously shows how lacking their system is.
Re: SREBench Competition
#14Re: SREBench Competition
#15But I think the bigger thing is... Your supposed AI not understanding answers that are literally identical to the ones it considers correct. Pretty weak.
Re: SREBench Competition
#16Re: SREBench Competition
#17Earlier quoted context omitted.
try to be as detailed as possible. With text sometimes it's hard to determine how close your answer was to the correct one.
How did a user with 3 strawberries as their name achieve a 3333333.33% success rate? EDIT: After asking this instead of responding back to me they just removed it. But it obviously shows how lacking their system is.
Re: SREBench Competition
#18What about your AI not being able to understand answers exactly identical to its own? Or the 3 strawberry user you had to remove that had a 33333% success rate you didn't remove until I said something. But I think the bigger thing is... Your supposed AI not understanding answers that are literally identical to the ones it considers correct. Pretty weak.
Re: SREBench Competition
#19Earlier quoted context omitted.
How did a user with 3 strawberries as their name achieve a 3333333.33% success rate? EDIT: After asking this instead of responding back to me they just removed it. But it obviously shows how lacking their system is.
the user competition is meant to be a fun side-project that we threw together today, I think it's cool that people hack things like that so quickly :)
The website comes off as a marketing strategy rather than a fun one-day hackathon project. I think that's why it's getting the reaction you're seeing.
Re: SREBench Competition
#20 Pod is stuck in 'ContainerCreating' state and never starts.
$ kubectl get po -A
```
NAMESPACE NAME READY STATUS RESTARTS AGE
default my-app-5d8d6f6d6f-abcde 1/1 Running 0 2d
default my-app-5d8d6f6d6f-fghij 1/1 Running 0 2d
kube-system coredns-558bd4d5db-xyz12 1/1 Running 0 5d
kube-system coredns-558bd4d5db-xyz34 1/1 Running 0 5d
kube-system etcd-minikube 1/1 Running 0 5d
kube-system kube-apiserver-minikube 1/1 Running 0 5d
kube-system kube-controller-manager-minikube 1/1 Running 0 5d
kube-system kube-proxy-abcde 1/1 Running 0 5d
kube-system kube-scheduler-minikube 1/1 Running 0 5d
kube-system storage-provisioner 1/1 Running 0 5d
```
Your root cause: no pod is stuck in ContainerCreating?
Grade: incorrect
My other problems were similarly confounding¹. One was "one machine seems loaded, but not others." All the pods had a node affinity to a single node tacked onto their specs, but that's only "partially correct"? And the last one is "Application components in different pods cannot communicate", but nothing is running except nginx, which would never communicate with itself.We're generating the problems, and answers, with an AI, aren't we?
I've thrown a few real-world problems at LLMs, and they have floundered on them, to the point of not even being able to emit coherent output. I've had utterly incoherent responses, "add this label to the pod label is in Chinese", etc.
Edit: played again. Got the same node affinity problem. Same answer, but this time it was correct. Oh yeah, AI comin' for my job /s.
Also no alias k=kubectl and no up/down to repeat/edit commands, the site restricts you from copy/pasting pod names (or anything else), no tab complete, no common shortcuts… — like yeah, if this is the condition your SREs are working in then I bet an AI can beat them? Might as well tie their hands behind their backs while we're at it.
¹I suppose it matches real life, in that the reported problem is often utterly divorced from reality, and it takes 2–3 rounds with the reporter to make sense of what it is they're trying to report in the first place. But I can't interrogate the problem statement in this "simulator".