Conversation

Any recommendations for long term metric storage?

I tweaked Prometheus to store a year's worth of data, but one unclean shutdown and I've got a corrupt WAL block, which it doesn't handle well (needs 20GiB RAM at startup).

There are many options (, , , ...), any reasonable low-maintenance / low-resource storage which I could write from Prometheus and show in dashboards?

3
0
0
@michal zkousel jsem Influx, ale nakonec skoncil u Timescale, protoze Postgres umim rozhodne lip adminovat
se zbytkem zkusenosti nemam
1
0
1

@michal uses S3 for persistence. uses local block storage, but has replication options at better performance than vanilla .

1
0
0

@claus I'd like to keep everything local, I'm certainly not going to pay to store some cpu load and temperature/humidity data :)

0
0
0

@albi https://github.com/timescale/promscale/issues/1836 "We regret to inform you that we’ve made the difficult decision to discontinue Promscale" :(

0
0
0

In the end I settled on Grafana Mimir: https://github.com/grafana/mimir

Deployment was easy (reasonable looking config file + Docker arm64 image), local storage to a single directory, reasonable migration story (tool which straight uploaded Prometheus data blocks, backfilled a year's worth of data), Grafana support as data source (duh!).

The only nuisance was that I had to modify every single graph to switch to the new data source...

0
0
0