Skip to main content

3 posts tagged with "InfluxDB"

View All Tags

Introducing ExyData Labs: Query to InfluxDB in natural language

· 3 min read

ExyData Labs

Introduction

Part of our DNA at ExyData is to always be at the forefront of technology. We are always looking for new ways to improve introduce tools that helps to improve our services and make them more accessible to our customers. That's why we are excited to introduce ExyData Labs, a new initiative to bring the power of AI / Machine Learning to ExyData services.

Our first project is a natural language interface to query InfluxDB using Flux. This new tool will allow you to ask questions in plain English and get answers from your InfluxDB database. No need to learn a new query language or spend hours writing complex queries. Just ask a question and get an answer.

How it works

For now, we have a prototype of what we called Flux GPT, that you can access from here: ExyData Labs. You can ask questions like "What is the average temperature in the last 24 hours?" or "Show me the top 5 servers by CPU usage". The tool will then translate your question into a Flux query and execute it against our InfluxDB database that we are filling with synthetic data.

Things to consider

This is a prototype and there are still many things to improve. The tool is still learning and may not always understand your questions or doesn't have the best performance. So, be patient :D

How behind the scenes looks like for this project?

Basically, what we did was to train a Llama model with a dataset of questions and their corresponding Flux queries. We then fine-tuned the model with our own dataset and deployed it to a web interface. The model is running on a server and receives the questions from the web interface, translates them into Flux queries, and executes them against our InfluxDB database.

The architecture, looks like this:

Conclusion

We are excited about the potential of this new tool and we hope that it will help you to get more insights and make it accessible to more people. We are planning to continue working on this project and improve it over time. Is not yet decided if we will release it as a product, but we are open to your feedback and suggestions. So, feel free to try it and let us know what you think.

How to Create a Status Dashboard with InfluxDB, Grafana, and Telegraf on ExyData

· 5 min read

Introduction

A few days ago we released our status page on ExyData. We want to create that glass panel that shows you our the infrastructure that we use to provide services is doing. In this tutorial, we will show you how to create a status dashboard with InfluxDB, Grafana, and Telegraf on ExyData.

Prerequisites

Before you start, you need to have the following prerequisites:

  • An ExyData account, you can create one for free here.
  • An InfluxDB instance, you can follow instructions on how to create one in our docs here.
  • A Grafana instance, you can follow instructions on how to create one in our docs here.
  • A Telegraf that you can run in your local or other server. The cool part about Telegraf is super small and easy to install. You can follow instructions on how to install it in InfluxData Docs here.

The Architecture

The architecture of the status dashboard is simple. We will use Telegraf to collect data from our infrastructure and send it to InfluxDB. Then, we will use Grafana to visualize the data in real-time.

Here is a diagram of the architecture:

Step 1: Install Telegraf

The first step is to install Telegraf on your local or other server. You can follow the instructions on how to install Telegraf in the InfluxData Docs here.

Step 1a: Configure Telegraf

This is the configuration that we are using for Telegraf and it is working well for us. You can copy and paste it into your telegraf.conf file.

warning

You need to replace the placeholders $INFLUXDB_URL, $INFLUXDB_DATABASE, $INFLUXDB_USERNAME, and $INFLUXDB_PASSWORD with your InfluxDB URL, database, username, and password. Also make sure to replace the URLs in the [[inputs.http_response]] section with the URLs you want to monitor.

# Telegraf Configuration

# Global Agent Configuration
[agent]
  interval = "1m" # How often to gather metrics
  round_interval = true
  metric_batch_size = 1000
  metric_buffer_limit = 10000
  collection_jitter = "0s"
  flush_interval = "1m"
  flush_jitter = "0s"
  precision = ""
  debug = false
  quiet = false
  logfile = ""  # Use "" to write to stderr

# Output Plugins Configuration
[[outputs.influxdb]]
  urls = ["$INFLUXDB_URL"]
  database = "$INFLUXDB_DATABASE"
  username = "$INFLUXDB_USERNAME"
  password = "$INFLUXDB_PASSWORD"

# Input Plugins Configuration

# HTTP Input Plugin to monitor HTTP response
[[inputs.http_response]]
  urls = ["https://exydata.com", "https://api.exydata.com/docs", "https://cloud.exydata.com"]
  method = "GET"
  follow_redirects = true

Step 1b: Start Telegraf

After you have configured Telegraf, you can start it by running the following command:

sudo systemctl start telegraf

Step 2: Deploy InfluxDB

The next step is to deploy InfluxDB. You can follow the instructions on how to deploy InfluxDB in our docs here.

Using the CLI, you can create a database and a user with the following commands:

exy deploy \
    --database influxdbv1 \
    --cloud aws \
    --region us-east-1 \
    --plan hobbyst \
    --org_id <your-org-id> \
    --token <your-access-token>

Once is deployed and your changed your password, how to do it here you are ready to replace the placeholders in the Telegraf configuration file with your InfluxDB URL, database, username, and password.

Step 3: Deploy Grafana

The final step is to deploy Grafana. You can follow the instructions on how to deploy Grafana in our docs here.

exy deploy \
    --database grafana \
    --cloud aws \
    --region us-east-1 \
    --plan hobbyst \
    --org_id <your-org-id> \
    --token <your-access-token>

After you have deployed Grafana, you can access it by going to the URL provided by ExyData and logging in with the credentials that we sent you to your email.

Step 3a: Configure InfluxDB Data Source

The first thing you need to do is to configure the InfluxDB data source in Grafana. You can follow the instructions on how to configure the InfluxDB data source in our docs here.

Step 4: Create a Status Dashboard

Now that you have Telegraf collecting data from your infrastructure and sending it to InfluxDB, and you have Grafana configured to visualize the data, you can create a status dashboard. Our dashboard looks like this and you can access it here.

Status Dashboard

This dashboard can be a starting point for you to create your own status dashboard. You can customize it by adding more panels, changing the colors, and adding alerts if you want.

You can download the JSON file of our dashboard here and import it into your Grafana instance to get started.

Conclusion

In this tutorial, we showed you how to create a status dashboard with InfluxDB, Grafana, and Telegraf on ExyData. You learned how to collect data from your infrastructure using Telegraf, store it in InfluxDB, and visualize it in Grafana. You can use this dashboard to monitor your data infrastructure in real-time and make informed decisions.

If you have any questions or feedback, feel free to reach out to us on through our contact form. We would love to hear from you!

Try ExyData Today and Get $250 in Free Credits!

· 4 min read

ExyData

Photo by Choong Deng Xiang on Unsplash

Introduction

When I started to think about ExyData, I had a clear vision in mind: If anybody wants to try the platform, they should be able to do so without any barriers. That's why I'm excited to announce that we're offering $250 in free credits to new users who sign up for ExyData today!

What is ExyData?

But first, in the case that is the first time that you are reading this blog or you are not familiar with ExyData, let me give you a brief overview of what ExyData is all about.

ExyData is a powerful platform that can help you tackle a wide range of data challenges, from manufacturing operations to fraud detection and energy management. By leveraging cutting-edge technologies like ClickHouse, InfluxDB, RabbitMQ, Grafana, and Superset, you can collect, analyze, and visualize data to make informed decisions and drive business growth. Whether you're looking to optimize your manufacturing processes, enhance security, or reduce energy costs, ExyData has you covered.

How to Get Started

To get started with ExyData, simply sign up for an account on our website and you'll automatically receive $250 in free credits. These credits can be used to explore the platform, run experiments, and build prototypes without any cost to you. Once you've used up your credits, you can choose to continue using ExyData by purchasing additional credits on demand or signing up for a subscription plan with invoice support and priority support.

We have a dedicated team of engineers and data scientists who are ready to help you get the most out of ExyData. Whether you need assistance with setting up your data pipelines, building machine learning models, or creating custom dashboards, we're here to support you every step of the way.

Using ExyData Web Console

The ExyData web console is a powerful tool that allows you to manage your data infrastructure, monitor your data pipelines, and visualize your data in real-time. With a user-friendly interface and a wide range of features, the web console makes it easy to get up and running with ExyData in no time. Is the more easy way to interact with the platform for non-technical users. You can signup for free and start using it today!

Using ExyData CLI

For more advanced users, we also offer a command-line interface (CLI) that allows you to interact with ExyData from the command line. With the CLI, you can automate tasks, manage your data infrastructure, and run experiments with ease. You can install the CLI by running the following command:

First, visit the releases page on the ExyData GitHub repository and download the latest release for your operating system. Once you've downloaded the CLI, you can register yourself in the platform by running the following command:

exy register --first_name "John" --last_name "Doe" --email "johndoe@yourdomain.com"

After you've registered, you can log in to the platform by running the following command:

exy login --email "johndoe@yourdomain.com" --password "yourpassword"

Once you're logged in and you have your auth token, you can start interacting with the platform. Check here the full documentation.

Conclusion

We can't wait to see what you'll build with ExyData! Whether you're a data scientist, a data engineer, or a business analyst, ExyData has the tools you need to succeed. Sign up today and claim your $250 in free credits to get started. We look forward to working with you and helping you achieve your data goals!

If you have any questions or need assistance, please don't hesitate to reach out to us at https://exydata.com/contact. Our team is here to help you every step of the way.