TL;DR Summary: Common ElasticSearch curl commands simplify data management tasks such as indexing, querying, and updating documents. These commands allow users to interact with the ElasticSearch RESTful API efficiently.
- ElasticSearch supports dynamic field mapping, enabling flexible data exploration.
- Commands include creating, deleting, and listing indices, as well as querying documents with URL parameters or JSON.
Here we show some of the most common ElasticSearch curl commands. ElasticSearch is sometimes complicated. So here we make it simple.
Background information
What is “curl”?
The acronym cURL, now frequently spelled curl, stands for client URL. It is a command line tool that lets you transfer data to or from web servers, APIs, and other networked devices. Developers find that it is an essential tool for use in web development projects, for automation and scripting tasks, and for managing IT systems.
Using ElasticSearch curl commands
For large-scale data indexing and querying, curl ElasticSearch is a powerful distributed search and analytics engine. You can directly access the ElasticSearch RESTful API using curl to create indexes, search through documents, update data, and for managing cluster computing. Here is a curl ElasticSearch commands cheat sheet:
ElasticSearch indices commands using curl
An ElasticSearch index is a set of signs that identify and refer to a set of documents that are a collection of fields, all holding data. What makes it elastic, as opposed to a fixed schema, is that you don’t have to map things out in advance. It lets you explore your data without requiring you to define your fields. It automatically detects and maps your fields dynamically.
Create index in ElasticSearch
The ElasticSearch command below creates an index named index_name with default settings.
curl -X PUT 'http://localhost:9200/index_name'
Delete an ElasticSearch index
To delete an ElasticSearch index (in this case, named samples), use the following curl command:
curl -X DELETE 'http://localhost:9200/samples'
List all indices
To list all indices in ElasticSearch, use:
curl -X GET 'http://localhost:9200/_cat/indices?v'
List all docs in index
Below is an curl ElasticSearch query to return all documents from an index:
curl -X GET 'http://localhost:9200/sample/_search'
List index mapping
All ElasticSearch fields are indexes. So this ElasticSearch curl command lists all fields and their types in an index.
curl -X GET http://localhost:9200/samples
Back up an ElasticSearch index
curl -XPOST --header 'Content-Type: application/json' http://localhost:9200/_reindex -d '{
"source": {
"index": "samples"
},
"dest": {
"index": "samples_backup"
}
}'
How to query ElasticSearch using curl
When you query ElasticSearch using curl, you can use URL parameters or send an HTTP GET request with a JSON body that contains the query. Here are examples of both approaches:
ElasticSearch curl query using URL parameters
Below is a simple curl to get an ElasticSearch curl command example, useful for basic queries and operations. Here we use Lucene query format to write q=school:Harvard.
curl -X GET http://localhost:9200/samples/_search?q=school:Harvard
Query with JSON aka ElasticSearch Query DSL
You can query using parameters on the URL. But you can also use JSON to query ElasticSearch using curl, as shown in the next example. JSON would be easier to read and debug when you have a complex query than one giant string of URL parameters.
curl -XGET --header 'Content-Type: application/json' http://localhost:9200/samples/_search -d '{
"query" : {
"match" : { "school": "Harvard" }
}
}'
ElasticSearch query returning only certain fields
To return only certain fields when querying ElasticSearch using curl, put them into the _source array:
GET filebeat-7.6.2-2020.05.05-000001/_search
{
"_source": ["suricata.eve.timestamp","source.geo.region_name","event.created"],
"query": {
"match" : { "source.geo.country_iso_code": "GR" }
}
}
ElasticSearch query by date
When the field is of the type “date”, you can use date math, like this:
GET filebeat-7.6.2-2020.05.05-000001/_search
{
"query": {
"range" : {
"event.created": {
"gte" : "now-7d/d"
}
}
}
}
Curl put ElasticSearch commands
Add data
The curl command below adds data to an ElasticSearch document:
curl -XPUT --header 'Content-Type: application/json' http://localhost:9200/samples/_doc/1 -d '{
"school" : "Harvard"
}'
Update doc
Here is how to add fields to an existing ElasticSearch document. First, we create a new doc, then we update it.
curl -XPUT --header 'Content-Type: application/json' http://localhost:9200/samples/_doc/2 -d '
{
"school": "Clemson"
}'
curl -XPOST --header 'Content-Type: application/json' http://localhost:9200/samples/_doc/2/_update -d '{
"doc" : {
"students": 50000}
}'
Other useful ElasticSearch curl command examples
Bulk load data in JSON format
Here’s how to bulk load data in JSON format to an ElasticSearch cluster:
export pwd="elastic:"
curl --user $pwd -H 'Content-Type: application/x-ndjson' -XPOST 'https://58571402f5464923883e7be42a037917.eu-central-1.aws.cloud.es.io:9243/0/_bulk?pretty' --data-binary @
Show ElasticSearch cluster health
The curl xget ElasticSearch command below uses JSON format to check ElasticSearch cluster health:
curl --user $pwd -H 'Content-Type: application/json' -XGET https://58571402f5464923883e7be42a037917.eu-central-1.aws.cloud.es.io:9243/_cluster/health?pretty
Aggregation and bucket aggregation
For an nginx web server, this ElasticSearch aggregation command produces web hit counts by user city:
curl -XGET --user $pwd --header 'Content-Type: application/json' https://58571402f5464923883e7be42a037917.eu-central-1.aws.cloud.es.io:9243/logstash/_search?pretty -d '{
"aggs": {
"cityName": {
"terms": {
"field": "geoip.city_name.keyword",
"size": 50
}
}
}
}
'
This expands that to a product response code count by city in an nginx web server log:
curl -XGET --user $pwd --header 'Content-Type: application/json' https://58571402f5464923883e7be42a037917.eu-central-1.aws.cloud.es.io:9243/logstash/_search?pretty -d '{
"aggs": {
"city": {
"terms": {
"field": "geoip.city_name.keyword"
},
"aggs": {
"responses": {
"terms": {
"field": "response"
}
}
}
},
"responses": {
"terms": {
"field": "response"
}
}
}
}'
Using ElasticSearch with basic authentication
If you have turned on security with ElasticSearch, then you need to supply the user and password as shown below to every curl command:
curl -X GET 'http://localhost:9200/_cat/indices?v' -u elastic:(password)
Pretty print ElasticSearch
Add ?pretty=true to any search to pretty print the JSON, like this:
curl -X GET 'http://localhost:9200/(index)/_search'?pretty=true
ElasticSearch Guide
- ElasticSearch Tutorial for Beginners: ElasticSearch Basics
- Using Apache Hive with ElasticSearch
- Using ElasticSearch with Apache Spark
- Curl ElasticSearch Commands Cheat Sheet
- Spark ElasticSearch Hadoop Update and Upsert Example and Explanation
- How To Write Apache Spark Data to ElasticSearch Using Python
- ElasticSearch Search Syntax and Boolean and Aggregation Searches
- ElasticSearch Nested Queries: How to Search for Embedded Documents
- ElasticSearch Joins: Has_Child, Has_parent query
- How to Setup up an Elastic Version 7 Cluster
- How to Configure Filebeat for nginx and ElasticSearch
- Using Beats and Logstash to Send Logs to ElasticSearch
- How to Load CSV File into ElasticSearch with Logstash
- ElasticSearch Aggregations Explained
- What is the ELK Stack?
- Logstash 101: Using Logstash in a Data Processing Pipeline
- Using Kibana to Execute Queries in ElasticSearch using Lucene and Kibana Query Language
- ElasticSearch Machine Learning
- How to Install Elastic Enterprise Search
- How To Use Elastic Enterprise Search with GitHub
These postings are my own and do not necessarily represent BMC's position, strategies, or opinion.