Want to dive deeper into #Grafana, #Prometheus #OpenTelemetry, or datasource plugins? 
Our hands-on labs at #GrafanaCON are where learning gets real. Tinker, test, and troubleshoot alongside Grafana experts. 
May 6 | Only with a GrafanaCON ticket
Add your labs when you register: https://grafana.com/events/grafanacon/2025/hands-on-labs/?src=masto&mdm=social&camp=gcon2025
This new Asteroid Hauling activity is not fun at all! It needs some major fine tuning! The ship doesn't need to be listing so drastically...The gravity Stabilizer should be helping out with that more. Either make the tilts less drastic or allow adjustment by more than one unit in the opposite direction at a time, probably a bit of both. I tried three times and got further the third time but only by speeding my speech way up and trying frantically to keep myself level but it still got away from me. The output is too verbose combined with it introducing roundtime randomly per each correction. I thought it might be based on the damage of the Gravity Stabilizer or another component, but that doesn't seem to be the case as far as I can tell. There's challenging and then there's ridiculously challenging... #Prometheus
If you've tried both Thanos and Mimir, which do you prefer? Feel free to comment why below 
So, I've been using Thanos to receive and store my prometheus metrics long term in a self hosted S3 bucket. Thanos also acts as a datasource for my dashboards in Grafana, and provides a Ruler, which evaluates alerting rulers and forwards them to my alertmanager. It's ok. It's certainly got it's downsides, which I can go into later, but I've thinking... what about Mimir?
How do you all feel about Grafana's Mimir (source on GitHub)? It's AGPL and seems to literally be a replacement of Thanos, which is Apache 2.0.
Thanos description from their website:
Open source, highly available Prometheus setup with long term storage capabilities.
Mimir description from their website:
...open source software project that provides horizontally scalable, highly available, multi-tenant, long-term storage for Prometheus and OpenTelemetry metrics.
Both with work with alloy and prometheus alike. Both require you to configure initially confusing hashrings and replication parameters. Both have a bunch of large companies adopting them, so... now I feel conflicted. Should I try mimir? Poll in reply.
Aaaaaah pushgateway you heathens whyyyyy #prometheus #kubecon
@erincandescent If a thing's worth doing, then it's worth over-doing. I'm now running the #Prometheus #Chrony exporter on 5 systems now, and using the error metric from its GH page, it looks like I'm running 50-100 usec of #NTP error now, which should be more than good enough.
Very happy to discover how easy it is to extend `node_exporter` with custom metrics:
Add `--collector.textfile.directory=/some/place` to the daemon's arguments (e.g. via ARGS in `/etc/default/prometheus-node-exporter`), and write `*.prom` files in that directory with your favorite tool, and **boom!**
Text format doc with a very good example: https://prometheus.io/docs/instrumenting/exposition_formats/
Creating Grafana alerts is something I'm less comfortable with, but I'm starting to get the hang of it…
Another #Grafana UI improvement. I've already spent some time finding the latest place for data sources, but it's still some work to add the most popular data source, #Prometheus.
Interested in #containers, #docker, #OCI and #guix #declarative #configuration?
Watch Giacomo Leidi's talk about self-hosting @forgejo using Guix's container backed configuration. Check it out 
His Gocix project shows how to bring together container-based software while benefiting from the resilience of declarative configuration! He has services for #prometheus #grafana #traefik #bonfire and more!
WildFly 36.0.0.Beta1 is now available at https://wildfly.org/downloads/ and in Maven. #Micrometer metrics via #Prometheus, #WildFly apps with #JBang and more! Details are at https://bit.ly/4jachOl
I'm going to be at #kubecon. At the maintainers summit beforehand, at the contribfest, and at the #headlamp project pavilion.
Contribfest session: https://kccnceu2025.sched.com/event/1td0n
I'm looking forward to connecting with folks working on different projects. People have been quite busy building out Headlamp Kubernetes UIs for ecosystem tooling and standards like #gatewayapi #prometheus #keda #flux #minikube #backstage #inspektorgadget #flagger and #certmanager
Want #cloudnative but with the power of #declarative configuration? The recoverability of #transactions for system configuration?
Wednesday it's the online #guix meet-up! With a great talk by @paulbutgold
about running docker / oci containers using the Guix configuration system.
His Gocix project has #prometheus, #grafana, #forgejo, #conduit and #traefik examples.
Meet-up details:
every now and then I'm like "I want to learn a new metrics/timeseries db" ... then I start doing that and I'm like "ughhh, I've been doing this in at least 6 different ways over the last 15 years, why do it again"
I really miss Graphite
It's the #DayOfHelios / Sol's Day / #Sunday! 
"#Zeus then sent #Hermes to Atreus and told him to get Thyestes to agree that Atreus should rule [Argos], if #Helios (the Sun) should journey backwards. Thyestes agreed, and Helios put his setting where he usually rose."
Pseudo-Apollodorus, Bibliotheca E2.12
#Prometheus, the creation of man, Roman relief
This week we talked about the #mythology of the #constellation #Centaurus
While the main story is about the centaur Chiron, his story also intersects with Prometheus, who was punished by Zeus to have his liver eaten by an Eagle every day... truly nightmare fuel 
To learn more about these #myths, check out our #podcast episode here: https://starrytimepodcast.podbean.com/e/centaurus-myths-and-retconstellations/
Man Prometheus is a pain to recover once its data store is in any way out of shape. Did NOT help that it was buried inside Kubernetes inside a PVC.
Thankfully it was only Dev environment today but if this ever pages on Prod we're losing data as it stands.
I'll write something up for a run book but eesh.
@andreasdotorg @redknight
I think at that level it's conceptually easy, you "just" need (wo-)manpower to set up and maintain everything yourself. Assuming you want to set up a new cloud provider from scratch and build one/two/three new DCs in different regions in Europe:
- buy standard "off-the-shelve" server hardware
- at this level you can use US networking equipment (firewalls, routers, switches)
- and then use/self-host all the open-source software you want
E.g.:
- use your favourite #Linux distro (#debian, #ubuntu, #fedora, or whatever)
- set up Netbox or a similar tool (and maybe phpIPAM) + #PostGreSQL Server
- there's probably no way around #OpenStack either way, with #MariaDB and some other open source tools in the background
- you can set up #Prometheus, #Grafana, #OpenSearch for observability
And on top of that offer services as you see fit:
- automate setup/maintenance of #Kubernetes clusters (I heard #RKE2 is a fairly self-contained #K8s distribution)
- automate setup/maintenance of DB servers
- provide a way to run "serverless" apps
- set up #nextcloud or so
Sometimes I find my odd humour in old code and comments and have a chuckle.
Today it was that I had aliased the Prometheus Community Helm repository to "prom-com" locally.
arc) 