Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Is anyone using neural networks for anomaly detection in observability? If so, which model and how many metrics are you supporting per core?


LSTM is common for this.

also https://facebook.github.io/prophet/


How data hungry is it, or what is the minimum volume of data needed before its worth investigating?


The more complex the data is, the more you need. If your values are always 5, then you need only one data point.


If your values were always 5,you wouldn't use an LSTM to model it either. So presumably there's a threshold for when LSTM becomes practical and useful, no?


Sure, that was an extreme example. The point was that the minimum of data is 1 point, maximum is "all of it". It entirely depends on your use case.


What do you mean by “observability”?


Telemetry. Dashboards. The application is knowing when a signal is anomalous.

https://en.wikipedia.org/wiki/Observability_(software)


Oh, yes I am working on that. Usually LSTM, exploring encoder-decoders and generative models, but also some simpler models based on periodic averages (which are surprisingly useful in some use cases). But I don’t have per-core metrics.


Depending on how stable your signal is, I've had good experience with seasonal ARIMA and LOESS (but it's not neural networks)




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: