Skip to content

Commit f9054d9

Browse files
Snuffy2Snuffy2
authored
Merge pull request #187 from Snuffy2/Change-recorder_prefilter-from-submodule
Change recorder_prefilter from submodule to subtree
2 parents 7c9de27 + 58ef6d6 commit f9054d9

5 files changed

Lines changed: 328 additions & 1 deletion

File tree

‎custom_components/places/recorder_history_prefilter‎

Submodule recorder_history_prefilter deleted from 9a1e142

‎custom_components/places/recorder_history_prefilter/.gitattributes‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
# Auto detect text files and perform LF normalization
2+
* text=auto

‎custom_components/places/recorder_history_prefilter/LICENSE‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
MIT License
2+
3+
Copyright (c) 2023 gcobb321
4+
5+
Permission is hereby granted, free of charge, to any person obtaining a copy
6+
of this software and associated documentation files (the "Software"), to deal
7+
in the Software without restriction, including without limitation the rights
8+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9+
copies of the Software, and to permit persons to whom the Software is
10+
furnished to do so, subject to the following conditions:
11+
12+
The above copyright notice and this permission notice shall be included in all
13+
copies or substantial portions of the Software.
14+
15+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21+
SOFTWARE.

‎custom_components/places/recorder_history_prefilter/README.md‎

Lines changed: 123 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,123 @@
1+
# Home Assistant Recorder History Prefilter
2+
3+
The HA Recorder module was modified in HA 2023.6.0 to no longer allow a custom component to insert a sensor or other type of entity into the *_exclude_e* list that defined the entities that should not be added to the History database (home_assistant_v2.db).
4+
5+
This module fixes that problem.
6+
7+
8+
9+
## How it Works
10+
11+
#### The Home Assistant Recorder Component
12+
13+
When Home Assistant starts, it loads the recorder component. The recorder component gets it's parameters from the HA configuration file (configuration.yaml) and builds the list of entities, globs, etc. to be filtered. It then starts a state change listener to call the filter module whenever an entity's state is changed. The filter determines if the entity should be added to the History database,
14+
15+
For efficiency, Home Assistant builds the filter that is checked one time when Home Assistant is loaded. The filter can not be changed or added to by a custom_component.
16+
17+
#### The Recorder History Prefilter
18+
19+
This module injects it's own filter (actually a copy of the recorder's filter) into the HA Recorder component, removes the HA listener that was set up by the HA Recorder and sets up a new state change listener. It is called when an entity's state changes instead of the HA Recorder filter and checks to see if the entity should be filtered. It the entity is not in it's list, the entity is passed to the HA Recorder filter module as it normally would be.
20+
21+
22+
23+
### Using the Recorder Prefilter
24+
25+
Entities are added or removed from this list using function calls from a custom component. More than one custom component can use this.
26+
27+
#### Adding Entities
28+
29+
The recorder prefilter is contained in the *recorder_prefilter.py* module.
30+
31+
- Download it and add it to your code base.
32+
33+
Import it into the your module that determines the entities to be filtered. This may be in *\__init__.py*, *sensor.py* or another module.
34+
35+
- `import recorder_prefilter`
36+
37+
The a single entity or a list of entities can be added at one time using the *add_filter* function call.
38+
39+
- `recorder_prefilter.add_filter(hass, 'icloud3_event_log')`
40+
- `recorder_prefilter.add_filter(hass, ['icloud3_event_log', 'icloud3_wazehist_track', 'gary_iphone_info'])`
41+
- `recorder_prefilter.add_filter(hass, [filtered_entities_list)`
42+
43+
Notes:
44+
45+
- The entity_type (*input_boolean, light, etc* ) needs to be used if it is a non-sensor entity.
46+
- The first custom component to use the *add_filter* function call will inject the *recorder_prefilter* into the HA Recorder component. All subsequent *add_filter* calls will update the filter list.
47+
48+
#### Removing Entities
49+
50+
Entities can be removed from the filter list using the *remove_filter* function call.
51+
52+
The a single entity or a list of entities can be removed at one time.
53+
54+
- `recorder_prefilter.remove_filter(hass, 'icloud3_event_log')`
55+
56+
- `recorder_prefilter.remove_filter(hass, ['icloud3_event_log', 'icloud3_wazehist_track', 'gary_iphone_info'])`
57+
58+
- `recorder_prefilter.remove_filter(hass, [filtered_entities_list)`
59+
60+
61+
62+
### Logging
63+
64+
Normal HA Logging is available for the *recorder_prefilter* module. Enable it in configuration.yaml as you would do with any other component.
65+
66+
logger:
67+
default: info
68+
logs:
69+
custom_components.icloud3.support.recorder_prefilter: info
70+
custom_components.places.recorder_prefilter: info
71+
72+
**Info logging** - Add basic records to the *home-assistant.log* file
73+
74+
```Recorder Prefilter Injection Started
75+
Recorder Prefilter Injection Started
76+
Recorder Prefilter Injection Completed
77+
Added Recorder Prefilter Entities (icloud3)-2
78+
Recorder Prefilter Entities Updated, Entities Filtered-2
79+
...
80+
...
81+
Added Recorder Prefilter Entities (places)-1
82+
Recorder Prefilter Entities Updated, Entities Filtered-3
83+
...
84+
...
85+
Added Recorder Prefilter Entities (icloud3)-6
86+
Recorder Prefilter Entities Updated, Entities Filtered-9
87+
```
88+
89+
90+
91+
**Debug logging** - Add operational records to the *home-assistant.log* file.
92+
93+
custom_components.icloud3.support.recorder_prefilter: debug
94+
custom_components.places.recorder_prefilter: debug
95+
96+
```Recorder Prefilter Injection Started
97+
Recorder Prefilter Injection Started
98+
Injecting Custom Exclude Entity Prefilter into Recorder
99+
Removing Recorder Event Listener
100+
Reinitializing Recorder Event Listener
101+
Recorder Prefilter Injection Completed
102+
Added Prefilter Entities-['icloud3_event_log', 'icloud3_wazehist_track']
103+
Added Recorder Prefilter Entities (icloud3)-2
104+
All Prefiltered Entities-['sensor.icloud3_event_log', 'sensor.icloud3_wazehist_track']
105+
Recorder Prefilter Entities Updated, Entities Filtered-2
106+
...
107+
...
108+
Added Prefilter Entities-sensor.gary_place
109+
Added Recorder Prefilter Entities (places)-1
110+
All Prefiltered Entities-['sensor.gary_place', 'sensor.icloud3_event_log', 'sensor.icloud3_wazehist_track']
111+
Recorder Prefilter Entities Updated, Entities Filtered-3
112+
...
113+
...
114+
Added Prefilter Entities-['sensor.gary_iphone_info', 'sensor.gary_iphone_trigger', 'sensor.lillian_iphone_info', 'sensor.lillian_iphone_trigger', 'sensor.lillian_watch_info', 'sensor.lillian_watch_trigger']
115+
Added Recorder Prefilter Entities (icloud3)-6
116+
All Prefiltered Entities-['sensor.gary_iphone_info', 'sensor.gary_iphone_trigger', 'sensor.gary_place', 'sensor.icloud3_event_log', 'sensor.icloud3_wazehist_track', 'sensor.lillian_iphone_info', 'sensor.lillian_iphone_trigger', 'sensor.lillian_watch_info', 'sensor.lillian_watch_trigger']
117+
Recorder Prefilter Entities Updated, Entities Filtered-9
118+
119+
```
120+
121+
122+
123+
Developed by: Gary Cobb, *iCloud3 iDevice Tracker custom component*, (aka geekstergary)

‎custom_components/places/recorder_history_prefilter/recorder_prefilter.py‎

Lines changed: 182 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,182 @@
1+
#<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
2+
#
3+
# HA RECORDER - EXCLUDE entities FROM BEING ADDED TO HISTORY DATABASE
4+
#
5+
#
6+
# The HA Recorder module was modified in HA 2023.6.0 to no longer allow a custom
7+
# component to insert entity entity names in the '_exclude_e' list that defined
8+
# entity entities to not be added to the History database (home_assistant_v2.db).
9+
#
10+
# This module fixes that problem by using a code injection process to provide a
11+
# local prefilter to determine if an entity should be added before the Recorder filter.
12+
#
13+
#
14+
# This injection has two methods:
15+
# add_filter - Add entities to the filter list
16+
# ----------
17+
# hass - HomeAssistant
18+
# entities to be filtered -
19+
# single entity - entity_id (string)
20+
# multiple entities - list of entity ids
21+
#
22+
# 'sensor.' will be added to the beginning of the entity id if
23+
# it's type is not specifid
24+
#
25+
# recorder_prefilter.add_filter(hass, 'filter_id1')
26+
# recorder_prefilter.add_filter(hass, ['filter_entity2', 'filter_entity3'])
27+
#
28+
#
29+
# remove_filter - Remove entities from the filter list
30+
# -------------
31+
# Same arguments for add_filter
32+
#
33+
# recorder_prefilter.remove_filter(hass, 'filter_id1')
34+
# recorder_prefilter.remove_filter(hass, ['filter_entity2', 'filter_entity3'])
35+
#
36+
#
37+
# Gary Cobb, iCloud3 iDevice Tracker, aka geekstergary
38+
#<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
39+
40+
from homeassistant.core import HomeAssistant
41+
from inspect import getframeinfo, stack
42+
import logging
43+
_LOGGER = logging.getLogger(__name__)
44+
45+
VERSION = 1.0
46+
47+
def add_filter(hass: HomeAssistant, entities=None):
48+
'''
49+
Inject the entity prefilter into the Recorder, remove Recorder listeners,
50+
reinitialize the Recorder
51+
52+
Arguments:
53+
hass - HomeAssistant
54+
entities - A list of entity entities
55+
(['gary_last_update', 'lillian_last_update', '*_next_update'])
56+
- A single entity entity ('gary_last_zone')
57+
58+
Returns:
59+
True - The injection was successful
60+
False - The injection was not successful
61+
'''
62+
63+
ha_recorder = hass.data['recorder_instance']
64+
65+
if ha_recorder is None:
66+
return False
67+
68+
if hass.data.get('recorder_prefilter') is None:
69+
rp_data = hass.data['recorder_prefilter'] = {}
70+
rp_data['injected'] = True
71+
rp_data['legacy'] = True
72+
rp_data['exclude_entities'] = []
73+
74+
try:
75+
ha_recorder.entity_filter._exclude_e.add(entities)
76+
return True
77+
except:
78+
pass
79+
80+
rp_data['legacy'] = False
81+
82+
if _inject_filter(hass) is False:
83+
return
84+
85+
_update_filter(hass, entities)
86+
87+
88+
def remove_filter(hass: HomeAssistant, entities):
89+
if hass.data['recorder_prefilter']['legacy']:
90+
try:
91+
ha_recorder = hass.data['recorder_instance']
92+
ha_recorder.entity_filter._exclude_e.discard(entities)
93+
return True
94+
except Exception as err:
95+
_LOGGER.exception(err)
96+
97+
_update_filter(hass, entities, remove=True)
98+
99+
100+
def _inject_filter(hass: HomeAssistant):
101+
ha_recorder = hass.data['recorder_instance']
102+
rp_data = hass.data['recorder_prefilter']
103+
recorder_entity_filter = ha_recorder.entity_filter
104+
recorder_remove_listener = ha_recorder._event_listener
105+
106+
def entity_filter(entity_id):
107+
"""
108+
Prefilter an entity to see if it should be excluded from
109+
the recorder history.
110+
111+
This function is injected into the recorder, replacing the
112+
original HA recorder_entity_filter module.
113+
114+
Return:
115+
False - The entity should is in the filter list
116+
Run the original HA recorder_entity_filter function -
117+
The entity is not in the filter list.
118+
"""
119+
if (entity_id
120+
and entity_id in hass.data['recorder_prefilter']['exclude_entities']):
121+
return False
122+
123+
return recorder_entity_filter(entity_id)
124+
125+
try:
126+
_LOGGER.info("Recorder Prefilter Injection Started")
127+
_LOGGER.debug("Injecting Custom Exclude Entity Prefilter into Recorder")
128+
ha_recorder.entity_filter = entity_filter
129+
130+
_LOGGER.debug("Removing Recorder Event Listener")
131+
recorder_remove_listener()
132+
133+
_LOGGER.debug("Reinitializing Recorder Event Listener")
134+
hass.add_job(ha_recorder.async_initialize)
135+
136+
_LOGGER.info(f"Recorder Prefilter Injection Completed")
137+
138+
return True
139+
140+
except Exception as err:
141+
_LOGGER.info(f"Recorder Prefilter Injection Failed ({err})")
142+
_LOGGER.exception(err)
143+
144+
return False
145+
146+
147+
def _update_filter(hass: HomeAssistant, entities=None, remove=False):
148+
""" Update the filtered entity list """
149+
150+
mode = 'Removed' if remove else 'Added'
151+
cust_component = _called_from()
152+
entities_cnt = 1 if type(entities) is str else len(entities)
153+
154+
_LOGGER.debug(f"{mode} Prefilter Entities ({cust_component})-{entities}")
155+
_LOGGER.info(f"{mode} Recorder Prefilter Entities "
156+
f"({cust_component})-{entities_cnt}")
157+
158+
entities = [entities] if type(entities) is str else \
159+
entities if type(entities) is list else \
160+
[]
161+
162+
163+
rp_data = hass.data.get('recorder_prefilter')
164+
rp_exclude_entities = rp_data['exclude_entities']
165+
166+
for entity in entities:
167+
if entity.find('.') == -1:
168+
entity = f"sensor.{entity}"
169+
if entity not in rp_exclude_entities:
170+
if remove is False:
171+
rp_exclude_entities.append(entity)
172+
elif entity in rp_exclude_entities:
173+
rp_exclude_entities.remove(entity)
174+
175+
_LOGGER.debug(f"All Prefiltered Entities-{sorted(rp_exclude_entities)}")
176+
_LOGGER.info(f"Recorder Prefilter Entities Updated, "
177+
f"Entities Filtered-{len(rp_exclude_entities)}")
178+
179+
180+
def _called_from():
181+
cust_component = getframeinfo(stack()[0][0]).filename
182+
return cust_component.split('custom_components/')[1].split('/')[0]

0 commit comments

Comments
 (0)