[ELK] Change raw mappings to support search fields #665

valeriocos · 2019-07-21T08:11:13Z

This PR showcases the effort on the Jira connector needed to change the mapping of the raw items, which would leverage on the search_fields metadata attribute.

Note data the test data should be recreated to include the search_fields attribute

coveralls · 2019-07-21T08:25:10Z

Pull Request Test Coverage Report for Build 2062

0 of 0 changed or added relevant lines in 0 files are covered.
18 unchanged lines in 3 files lost coverage.
Overall coverage increased (+0.05%) to 77.021%

Files with Coverage Reduction	New Missed Lines	%
/home/travis/build/chaoss/grimoirelab-elk/grimoire_elk/raw/github.py	2	95.0%
/home/travis/build/chaoss/grimoirelab-elk/grimoire_elk/raw/launchpad.py	6	0%
/home/travis/build/chaoss/grimoirelab-elk/grimoire_elk/raw/gitlab.py	10	52.38%

Totals
Change from base Build 2058:	0.05%
Covered Lines:	7260
Relevant Lines:	9426

💛 - Coveralls

This code changes the filter-raw parameter to `search_terms`. This change is needed to use the new Perceval metadata attribute `search_terms` to filter raw data. Signed-off-by: Valerio Cosentino <[email protected]>

This code changes the mapping of the Jira raw items. The new mapping is the following: ``` { "dynamic":true, "properties": { "data": { "dynamic":false, "properties": {} } } } ``` Signed-off-by: Valerio Cosentino <[email protected]>

This code changes the mapping of the raw items. The new mapping is the following: ``` { "dynamic":true, "properties": { "data": { "dynamic":false, "properties": {} } } } ``` Signed-off-by: Valerio Cosentino <[email protected]>

This code updates the expected results of the test test_all_properties. This change is needed to match the new mappings, which doesn't index the `data` attribute. Signed-off-by: Valerio Cosentino <[email protected]>

valeriocos mentioned this pull request Jul 21, 2019

[backend] Add search terms chaoss/grimoirelab-perceval#554

Closed

valeriocos added 3 commits March 31, 2020 12:54

[elastic_items] Change filter-raw parameter

aab4af9

This code changes the filter-raw parameter to `search_terms`. This change is needed to use the new Perceval metadata attribute `search_terms` to filter raw data. Signed-off-by: Valerio Cosentino <[email protected]>

[raw-jira] Change mapping

5c989fe

This code changes the mapping of the Jira raw items. The new mapping is the following: ``` { "dynamic":true, "properties": { "data": { "dynamic":false, "properties": {} } } } ``` Signed-off-by: Valerio Cosentino <[email protected]>

[raw-askbot] Change mapping

3ce862f

This code changes the mapping of the raw items. The new mapping is the following: ``` { "dynamic":true, "properties": { "data": { "dynamic":false, "properties": {} } } } ``` Signed-off-by: Valerio Cosentino <[email protected]>

valeriocos force-pushed the support-search-terms branch 2 times, most recently from 5175bb4 to 32d2ffd Compare March 31, 2020 12:22

valeriocos changed the title ~~[ELK] Change raw mappings to support search terms~~ [ELK] Change raw mappings to support search fields Mar 31, 2020

valeriocos added 22 commits March 31, 2020 14:34

[raw-bugzilla] Change mapping

3ae3458

This code changes the mapping of the raw items. The new mapping is the following: ``` { "dynamic":true, "properties": { "data": { "dynamic":false, "properties": {} } } } ``` Signed-off-by: Valerio Cosentino <[email protected]>

[raw-bugzillarest] Change mapping

256e4cc

This code changes the mapping of the raw items. The new mapping is the following: ``` { "dynamic":true, "properties": { "data": { "dynamic":false, "properties": {} } } } ``` Signed-off-by: Valerio Cosentino <[email protected]>

[raw-confluence] Change mapping

6c86f0e

This code changes the mapping of the raw items. The new mapping is the following: ``` { "dynamic":true, "properties": { "data": { "dynamic":false, "properties": {} } } } ``` Signed-off-by: Valerio Cosentino <[email protected]>

[raw-discourse] Change mapping

fdce2e2

This code changes the mapping of the raw items. The new mapping is the following: ``` { "dynamic":true, "properties": { "data": { "dynamic":false, "properties": {} } } } ``` Signed-off-by: Valerio Cosentino <[email protected]>

[raw-dockerhub] Change mapping

65f1c17

This code changes the mapping of the raw items. The new mapping is the following: ``` { "dynamic":true, "properties": { "data": { "dynamic":false, "properties": {} } } } ``` Signed-off-by: Valerio Cosentino <[email protected]>

[raw-gerrit] Change mapping

f16c193

This code changes the mapping of the raw items. The new mapping is the following: ``` { "dynamic":true, "properties": { "data": { "dynamic":false, "properties": {} } } } ``` Signed-off-by: Valerio Cosentino <[email protected]>

[raw-git] Change mapping

809f46a

This code changes the mapping of the raw items. The new mapping is the following: ``` { "dynamic":true, "properties": { "data": { "dynamic":false, "properties": {} } } } ``` Signed-off-by: Valerio Cosentino <[email protected]>

[raw-github] Change mapping

b7bd993

This code changes the mapping of the raw items. The new mapping is the following: ``` { "dynamic":true, "properties": { "data": { "dynamic":false, "properties": {} } } } ``` Signed-off-by: Valerio Cosentino <[email protected]>

[raw-gitlab] Change mapping

6895013

This code changes the mapping of the raw items. The new mapping is the following: ``` { "dynamic":true, "properties": { "data": { "dynamic":false, "properties": {} } } } ``` Signed-off-by: Valerio Cosentino <[email protected]>

[raw-googlehits] Change mapping

da494c0

This code changes the mapping of the raw items. The new mapping is the following: ``` { "dynamic":true, "properties": { "data": { "dynamic":false, "properties": {} } } } ``` Signed-off-by: Valerio Cosentino <[email protected]>

[raw-hyperkitty] Change mapping

e5877cd

This code changes the mapping of the raw items. The new mapping is the following: ``` { "dynamic":true, "properties": { "data": { "dynamic":false, "properties": {} } } } ``` Signed-off-by: Valerio Cosentino <[email protected]>

[raw-jenkins] Change mapping

b331084

This code changes the mapping of the raw items. The new mapping is the following: ``` { "dynamic":true, "properties": { "data": { "dynamic":false, "properties": {} } } } ``` Signed-off-by: Valerio Cosentino <[email protected]>

[raw-launchpad] Change mapping

4adfc39

This code changes the mapping of the raw items. The new mapping is the following: ``` { "dynamic":true, "properties": { "data": { "dynamic":false, "properties": {} } } } ``` Signed-off-by: Valerio Cosentino <[email protected]>

[raw-mattermost] Change mapping

769a1c7

This code changes the mapping of the raw items. The new mapping is the following: ``` { "dynamic":true, "properties": { "data": { "dynamic":false, "properties": {} } } } ``` Signed-off-by: Valerio Cosentino <[email protected]>

[raw-mbox] Change mapping

31f1a08

This code changes the mapping of the raw items. The new mapping is the following: ``` { "dynamic":true, "properties": { "data": { "dynamic":false, "properties": {} } } } ``` Signed-off-by: Valerio Cosentino <[email protected]>

[raw-mediawiki] Change mapping

4f23dc5

This code changes the mapping of the raw items. The new mapping is the following: ``` { "dynamic":true, "properties": { "data": { "dynamic":false, "properties": {} } } } ``` Signed-off-by: Valerio Cosentino <[email protected]>

[raw-meetup] Change mapping

738ee4d

This code changes the mapping of the raw items. The new mapping is the following: ``` { "dynamic":true, "properties": { "data": { "dynamic":false, "properties": {} } } } ``` Signed-off-by: Valerio Cosentino <[email protected]>

[raw-nntp] Change mapping

39d5063

This code changes the mapping of the raw items. The new mapping is the following: ``` { "dynamic":true, "properties": { "data": { "dynamic":false, "properties": {} } } } ``` Signed-off-by: Valerio Cosentino <[email protected]>

[raw-redmine] Change mapping

dbd7341

This code changes the mapping of the raw items. The new mapping is the following: ``` { "dynamic":true, "properties": { "data": { "dynamic":false, "properties": {} } } } ``` Signed-off-by: Valerio Cosentino <[email protected]>

[raw-phabricator] Change mapping

77e685a

This code changes the mapping of the raw items. The new mapping is the following: ``` { "dynamic":true, "properties": { "data": { "dynamic":false, "properties": {} } } } ``` Signed-off-by: Valerio Cosentino <[email protected]>

[raw-rss] Change mapping

65a0357

This code changes the mapping of the raw items. The new mapping is the following: ``` { "dynamic":true, "properties": { "data": { "dynamic":false, "properties": {} } } } ``` Signed-off-by: Valerio Cosentino <[email protected]>

[raw-slack] Change mapping

6db8454

This code changes the mapping of the raw items. The new mapping is the following: ``` { "dynamic":true, "properties": { "data": { "dynamic":false, "properties": {} } } } ``` Signed-off-by: Valerio Cosentino <[email protected]>

valeriocos added 13 commits March 31, 2020 14:34

[raw-stackexchange] Change mapping

9d587fb

This code changes the mapping of the raw items. The new mapping is the following: ``` { "dynamic":true, "properties": { "data": { "dynamic":false, "properties": {} } } } ``` Signed-off-by: Valerio Cosentino <[email protected]>

[raw-supybot] Change mapping

9066023

This code changes the mapping of the raw items. The new mapping is the following: ``` { "dynamic":true, "properties": { "data": { "dynamic":false, "properties": {} } } } ``` Signed-off-by: Valerio Cosentino <[email protected]>

[raw-telegram] Change mapping

dff989f

This code changes the mapping of the raw items. The new mapping is the following: ``` { "dynamic":true, "properties": { "data": { "dynamic":false, "properties": {} } } } ``` Signed-off-by: Valerio Cosentino <[email protected]>

[raw-twitter] Change mapping

cacb20e

This code changes the mapping of the raw items. The new mapping is the following: ``` { "dynamic":true, "properties": { "data": { "dynamic":false, "properties": {} } } } ``` Signed-off-by: Valerio Cosentino <[email protected]>

[raw-remo] Change mapping

25bcd9c

This code changes the mapping of the raw items. The new mapping is the following: ``` { "dynamic":true, "properties": { "data": { "dynamic":false, "properties": {} } } } ``` Signed-off-by: Valerio Cosentino <[email protected]>

[raw-finosmeetings] Change mapping

1cd233a

This code changes the mapping of the raw items. The new mapping is the following: ``` { "dynamic":true, "properties": { "data": { "dynamic":false, "properties": {} } } } ``` Signed-off-by: Valerio Cosentino <[email protected]>

[raw-functest] Change mapping

0609637

This code changes the mapping of the raw items. The new mapping is the following: ``` { "dynamic":true, "properties": { "data": { "dynamic":false, "properties": {} } } } ``` Signed-off-by: Valerio Cosentino <[email protected]>

[raw-crates] Change mapping

f54381c

This code changes the mapping of the raw items. The new mapping is the following: ``` { "dynamic":true, "properties": { "data": { "dynamic":false, "properties": {} } } } ``` Signed-off-by: Valerio Cosentino <[email protected]>

[raw-graal] Change mapping

93843e9

This code changes the mapping of the raw items. The new mapping is the following: ``` { "dynamic":true, "properties": { "data": { "dynamic":false, "properties": {} } } } ``` Signed-off-by: Valerio Cosentino <[email protected]>

[raw-kitsune] Change mapping

7e55380

This code changes the mapping of the raw items. The new mapping is the following: ``` { "dynamic":true, "properties": { "data": { "dynamic":false, "properties": {} } } } ``` Signed-off-by: Valerio Cosentino <[email protected]>

[raw-mozillaclub] Change mapping

f68c273

This code changes the mapping of the raw items. The new mapping is the following: ``` { "dynamic":true, "properties": { "data": { "dynamic":false, "properties": {} } } } ``` Signed-off-by: Valerio Cosentino <[email protected]>

[raw-puppetforge] Change mapping

8493671

This code changes the mapping of the raw items. The new mapping is the following: ``` { "dynamic":true, "properties": { "data": { "dynamic":false, "properties": {} } } } ``` Signed-off-by: Valerio Cosentino <[email protected]>

[tests] Update test_all_properties in test_elastic

04f2b38

This code updates the expected results of the test test_all_properties. This change is needed to match the new mappings, which doesn't index the `data` attribute. Signed-off-by: Valerio Cosentino <[email protected]>

valeriocos force-pushed the support-search-terms branch from 32d2ffd to 04f2b38 Compare March 31, 2020 12:34

valeriocos mentioned this pull request Apr 5, 2020

[gitter] add gitter backend support to ELK #831

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[ELK] Change raw mappings to support search fields #665

[ELK] Change raw mappings to support search fields #665

valeriocos commented Jul 21, 2019 •

edited

Loading

coveralls commented Jul 21, 2019 •

edited

Loading

[ELK] Change raw mappings to support search fields #665

Are you sure you want to change the base?

[ELK] Change raw mappings to support search fields #665

Conversation

valeriocos commented Jul 21, 2019 • edited Loading

coveralls commented Jul 21, 2019 • edited Loading

Pull Request Test Coverage Report for Build 2062

💛 - Coveralls

valeriocos commented Jul 21, 2019 •

edited

Loading

coveralls commented Jul 21, 2019 •

edited

Loading