Elasticsearch guide

Elasticsearch guide

Download pdf

Introduction

The Indicia development team has produced a new search technique which is the basis of the new Elasticsearch page. Not only does the speed up the searches but the page also has additional functions which are outlined below.

This article covers the following topics

  • Quick Overview
  • Associated organisms
  • Customising your page
  • Filtering and searching
    • Top bar
    • Create a filter
    • Column headers
  • Map settings
  • Examples

Overview

The highlighted areas link to the following new search and display tools.

  1. There is a general search box and dropdown filters for vice counties and local records centres
    1. The search box searches nearly all the database not just those visible on the page.
    2. Complex search commands can be entered here and this is summarised in a pop up which appears over the column head search boxes.
  2. The create a filter option is unchanged from the earlier explore records and works with the other search boxes and dropdowns on the page.
  3. This popup allows the background map to be changed and also select which map icons appear. These will be described later as they can be combined to show different data
  4. Tabs to change the grid from records to the species summary of filtered records.
  5. The user can select which columns appear on the page using the spanner icon in area 5.
    The column head search boxes allow free text searches of the column below and again most support complex text searches.
  6. There are two icons here
    1. The spanner icon brings up a popup to choose which columns appear on the page and order them.
    2. The grid icon makes just the grid appear in full width on the webpage.
  7. This area shows the total number of records found and buttons to page through the records.

Associated organisms

As any entered associated organism creates its own record the initial list of species in the grid includes both fungi and their associated organisms. Additionally, there is a column called Associations which also lists the same associations not just from fungi to plant but also plant to fungi. This provides additional search capabilities which will be described below.

Tip: You can use Create a filter to design and save a filter that looks for fungi, protozoa and chromista which will display just the fungi, slime moulds and downy mildews.

Customising your page
 

The columns displayed in the grid can be selected to suit the size of your computer. Apart from changing the number of columns you may wish to adjust the magnification of the page using your browser (Chrome, Edge, Safari etc) settings.

To select the columns click on the spanner icon in area 5 which brings up a picking list.

You can select / deselect any field using the tick box on the left of the field and rearrange the order of the fields by clicking and dragging the field up and down the list. Click save when you have finished selecting. Your selection will be remembered next time you log in. As before it is still possible to view the whole record in a new tab using the symbol at the far right of the record in the grid display.

Tip: Right click the icon and click Open in new tab in order to keep your grid of records visible.

The columns do resize and collapse on narrow screens which can be seen by clicking the + symbol on the left.

 

Note: There are three vice county fields. Vice County is calculated from the Grid reference. VCnum (ent) and VCname (ent) show the content entered when adding a record. 

Filtering using the top bar

The top bar currently has a general search box and two drop down menus.

The drop-down menus are for Vice Counties and Local Record Centres and provide a quick way to the select the records in these areas.

Although it is often easier to use the column head search boxes or the create a filter option, entering a term in the search box will search all the record fields in the database including many of the fields not visible in the grid. If the search term is not visible in the grid it will be visible when viewing the full record details. Eg if you have entered a collector’s ID it will be found.

Advanced: Apart from just entering a simple word it is possible to perform more complex searches using the following advice which is visible by hovering over many of column head search boxes. See example later.

"Search for words in the Location column. Prefix with ! to exclude rows which contain words beginning with the text you enter. Use * at the end of words to find words starting with. Use “” to group words into phrases and | between words to request either/or searches. Use - before a word to exclude that word from the search results."

It is also possible to limit which field is searched by specifying it first for example event.month:9 will search for all records from September. There is a fuller description of this on iRecord website here: https://www.brc.ac.uk/irecord/es-search-box which has further links to this very extensive facility.

Filtering using the Create a filter option

This option works in the same way as it does it does on the Explore all /my records webpages. There are a few aspects I would like to highlight.

  • Save the filter. Just a reminder that any filter you create can be saved for future use.
  • Viewing my records. In order to just view the records you have entered on the system select Who and tick the view my records box.
  • External key. Selecting Record ID now gives the option to enter either the record ID or and external ID. Currently most of the transferred FRDBI records can be found be entering the entering the previous FRDBI number here.

Filtering using the column headers

Entering a word or phrase in the column headers will just search that column for the phrase. Once again many of the fields can be searched using the more detailed methods described above for the top bar. If any filters have been employed in the drop downs at the top or the Create a filter options they will be combined with the column headings filters.

Map settings

The icon in the top right of the map allows various display options to be changed.

Firstly the background map can be changed from the basic street map to a topological option.

Secondly there are four on / off options for the following

  • Blue circles to show the distribution of records. These records can be filtered by the drop down, the create a filter and the column head searches. This provides a quick overview of searches and is the easiest approach when starting.
  • A blue blurred image of the distributions of records.  These records can be filtered using the drop downs and the create a filter BUT not the column head searches.
  • Red circles showing the distribution of records for any species selected in the records or species list. This is not filtered by the drop downs, the create a filter or other column heads.
  • Red blurred image of the distribution of records for any species selected in the records or species list. Again this is not filtered by the drop downs or the create a filter.

This might sound very complex but just using the first option allows a simple initial approach.

Examples

A number of examples will show some of the possible searches. First of all just select the top display option in the map icon drop down. (All records in current filter)

  1. Find all species of lachnum found on Juncus
    1. Reload the page to get rid of any previous searches
    2. Enter lachnum in the species column head
    3. Enter juncus in the associatons column head
    4. Click the Species tab to obtain the list and summary count of species
  2. Find all associated organisms for Boletus pinophilus
    1. Reload the page to get rid of any previous searches
    2. Enter “boletus pinophilus” in the associations column head
      (The quotes stop the search showing all boletus records)
    3. Click the Species tab to obtain the list and summary count of species
  3. Find all records for me or my friend Janke
    1. Reload the page to get rid of any previous searches
    2. Enter skeates|janke in the search box at the top.
      (The pipe symbol is found using the shift key and backslash on my keyboard.)
      This will find either name in either the recorder or determiner field.
  4. Find any heliotales on quercus leaves.
    1. Reload the page to get rid of any previous searches
    2. Go to Create a filter -> What -> higher taxa and enter helotiales in box and click drop down, click add and then apply
    3. Enter quercus in associations column header
    4. Enter leaf in substrate column header
    5. Click the Species tab to obtain the list and summary count of species
    6. The species names can be put n alphabetical order using the arrow by accepted name.

Finally here is an example using the display options of the map.

  1. Reload the page to get rid of any previous searches
  2. Click on the first display three options in the map options drop down.
  3. Enter boletus in the species column head box
  4. Click the Species tab
  5. Click Boletus armeniacus in the list

The map will now show a blue blurred image of all the records in the database, blue circles for all the boletus records found by the column head search and all the Boletus armeniacus records in red circles.
 

Future changes

This elastic search system is still being developed not just for us but all groups using the database warehouse. There are some tweaks I can make but others require the development team’s input. Your feedback and ideas will be welcome.

Stuart Skeates 2020