Conversation
| ;; | ||
| 8.0.1) | ||
| docker run --rm -d -p "${ES_PORT}:9200" \ | ||
| docker run --rm -d --name uptasticsearch -p "${ES_PORT}:9200" \ |
There was a problem hiding this comment.
wtf is going onnnnnnnnn
Elasticsearch did not become reachable within 1 minute
--- docker logs ---
Error response from daemon: No such container: uptasticsearch
There was a problem hiding this comment.
Ok so using docker run --rm meant the container wasn't there any more after it failed to come up, so docker logs couldn't show the logs.
Removing that, I was finally able to get some logs:
2026-09-02 02:10:35,533 main ERROR Could not reconfigure JMX java.lang.NullPointerException: Cannot invoke "jdk.internal.platform.CgroupInfo.getMountPoint()" because "anyController" is null
at java.base/jdk.internal.platform.cgroupv2.CgroupV2Subsystem.getInstance(CgroupV2Subsystem.java:81)
at java.base/jdk.internal.platform.CgroupSubsystemFactory.create(CgroupSubsystemFactory.java:113)
Exception in thread "main" java.lang.NullPointerException: Cannot invoke "jdk.internal.platform.CgroupInfo.getMountPoint()" because "anyController" is null
It looks like this was actually a bug in Java, related to Linux cgroups v2 and how java processes detect the amount of available memory when running in a container: https://bugs.openjdk.org/browse/JDK-8287073
There's a note buried at the bottom of https://www.elastic.co/support/matrix about this 🙃
Elasticsearch versions 8.0.0–8.4.3 and 7.15.1–7.17.6 fail to start on hosts using cgroup v2 with the bundled JDK.
We're just running a lil baby test instance here and don't need to care about getting fancy with container support, so after all that the fix is to pass -e "JDK_JAVA_OPTIONS=-XX:-UseContainerSupport" to tell the JDK not to try to do any smart container-y stuff.
What a ride!
| sleep 30 | ||
| # wait for the cluster to be reachable | ||
| echo "waiting for Elasticsearch to come up..." | ||
| SECONDS=0 |
There was a problem hiding this comment.
Learned about this while working on this tonight. I'd wanted to use timeout but that's one of those GNU things that isn't available by default on macOS.
From https://man7.org/linux/man-pages/man1/bash.1.html
Each time this parameter is referenced, it expands to the number of seconds since shell invocation. If a value is assigned to SECONDS, the value returned upon subsequent references is the number of seconds since the assignment plus the value assigned. The number of seconds at shell invocation and the current time are always determined by querying the system clock at one-second resolution. If SECONDS is unset, it loses its special properties, even if it is subsequently reset.
| ;; | ||
| 8.15.5) | ||
| docker run --rm -d -p "${ES_PORT}:9200" \ | ||
| docker run -d --name uptasticsearch -p "${ES_PORT}:9200" \ |
There was a problem hiding this comment.
Removing --rm allows docker logs use later if the container doesn't come up.
Giving it a stable name with --name uptasticsearch makes the docker logs command easier.
| echo "killing running container" | ||
| docker kill "$(docker ps -ql)" | ||
| docker kill -f uptasticsearch | ||
| docker rm -f uptasticsearch |
There was a problem hiding this comment.
The rm is needed because I dropped all the --rm from the docker run calls.
jameslamb
changed the title
|
Hey Austin check it out 😊 |
This branch has not been deployed
1 participant
Fixes #265
After seeing just that 8.0.1 job fail AGAIN on #272 I thought "ok, enough, let's just fix this".
See the inline comments for the investigation that led to these changes.
In short:
docker ps/docker logstosetup_local.shso we get more information when failures like this happen in the futuresetup_local.shto poll the Elasticsearch container until it's ready, which should allow us to tune timeouts to make this pass (and avoid a longsleepfor other versions that don't need it!).pre-commithooks withpre-commit autoupdateand fixes one finding from the latest version ofzizmor