race condition when updating dcc.Store

[logankopas]

Hello!

dash                 2.18.0
dash-core-components 2.0.0
dash-html-components 2.0.0
dash-table           5.0.0

• if frontend related, tell us your Browser, Version and OS

  • OS: macOS Sonoma
  • Browser: Tested in Firefox and Chrome
  • FF Version: 129.0.2
  • Chrome Version: 128.0.6613.121

Describe the bug

When 2 callbacks perform partial updates to a dcc.Store at the same time (or nearly the same time), only 1 of those updates is reflected in the store. I tested and found the same behaviour in dash versions 2.17.0 and 2.18.0, and this happens for all storage types (memory, session, and local).

A minimal example is below. Most of this example is setting up preconditions to cause the race condition, but it roughly matches our real-world use-case and can reliably exhibit the behaviour.

The example app works like this:

We have multiple components on the page that need to load, and each has 2 elements to manage: the content Div and the Loading indicator. We also have a dispatcher (Interval component + loading_dispatcher callback) that kicks off the loading of these components in chunks. For each component, the dispatcher first turns on the Loading indicator, which then triggers the content Div to load (load_component function), which then triggers the Loading indicator to stop (stop_spinner function). We also have a cleanup function (mark_loaded) that waits for the components to finish loading, then pushes data to the store about which components have loaded. Finally, the set_status function checks the store, and if all of the components have loaded it updates the status Div at the bottom to indicate everything is fully loaded.

Minimal Example

from dash import Dash, html,callback,Output,Input,State,no_update,dcc, MATCH, ALL, Patch, callback_context, clientside_callback
from dash.exceptions import PreventUpdate
import time
import random

app = Dash(__name__)

NUM_COMPONENTS = 21
STORAGE_TYPE = 'local'

slow_components = [
    html.Div([
        html.Div(children='loading...', id={'type': 'slow-component', 'index': i}),
        dcc.Loading(id={'type': 'slow-component-animation', 'index': i}, display='hide')
    ])
for i in range(NUM_COMPONENTS)]
app.layout = html.Div(
    slow_components +
    [
        html.Hr(),
        html.Div(id='status', children='not all loaded'),

        dcc.Interval(id='timer', interval=2000, max_intervals=10),
        dcc.Store(id='loading-data', data={}, storage_type=STORAGE_TYPE, clear_data=True),
    ]
)

@callback(Output({'type': 'slow-component-animation', 'index':ALL}, 'display'),
          Input('timer', 'n_intervals'), prevent_initial_call=True)
def loading_dispatcher(n):
    # Kicks off loading for 3 components at a time
    if n is None or n > NUM_COMPONENTS/3:
        raise PreventUpdate()
    output_list = [no_update] * NUM_COMPONENTS
    current_chunk_start = list(range(0,NUM_COMPONENTS, 3))[n-1]
    output_list[current_chunk_start:current_chunk_start+3] = ['show']*3
    return output_list

@callback(
    Output({'type': 'slow-component', 'index': MATCH}, 'children'),
    Input({'type': 'slow-component-animation', 'index': MATCH}, 'display'),
    State({'type': 'slow-component-animation', 'index': MATCH}, 'id'),
    State({'type': 'slow-component', 'index': MATCH}, 'children'),
    prevent_initial_call=True
)
def load_component(display, id_, current_state):
    # "Loads" data for 1 second, updates loading text
    if current_state == 'loaded':
        raise PreventUpdate()
    print(f'loading {id_['index']}, {current_state}')
    time.sleep(1)
    print(f'loaded {id_['index']}')
    return 'loaded'

@callback(
    Output({'type': 'slow-component-animation', 'index':MATCH}, 'display', allow_duplicate=True),
    Input({'type': 'slow-component', 'index': MATCH}, 'children'),
    prevent_initial_call=True
)
def stop_spinner(loading_text):
    # After loading, removes spinner
    if loading_text == 'loaded':
        return 'hide'
    return no_update


@callback(
    Output('loading-data', 'data', allow_duplicate=True),
    Input({'type': 'slow-component-animation', 'index': ALL}, 'display'),
    prevent_initial_call=True
)
def mark_loaded(components):
    # When a component is fully loaded, mark it as such in the data store
    print('checking if components are loaded')
    update_dict = {}
    for component in callback_context.triggered:
        if component['value'] == 'hide':
            component_id = callback_context.triggered_prop_ids[component['prop_id']]['index']
            print(f'component {component_id} loaded')
            update_dict[component_id] = 'loaded'
    patch = Patch()
    patch.update(update_dict)
    print(f'adding to data store: {update_dict}')
    return patch  # <- This is where the race condition happens. If 2 callbacks patch the store at the same time, only 1 of those patches is applied

@callback(
    Output('status', 'children'),
    Output('loading-data', 'data', allow_duplicate=True),
    Input('loading-data', 'data'),
    prevent_initial_call=True
)
def set_status(loading_data):
    # Once all components are loaded, update the status bar to show we are fully loaded
    print(f'{loading_data=}')
    if loading_data is None:
        return no_update, no_update
    if len(loading_data) == NUM_COMPONENTS:
        print('FULLY LOADED')
        return 'FULLY LOADED', {}
    return no_update, no_update


if __name__ == '__main__':
    app.run(debug=True)

Expected behavior

The app should load each component, and once they are finished the bottom text would update to say "FULLY LOADED".

The logs would also show that after each item is added to the store, the next time "loading_data=" is printed it would contain all of the component indices that have been added to the store. At the end of the logs we would see every number from 0-20 as a key in the loading_data dictionary.

Example (abbreviated):

loading 0, loading...
loading 1, loading...
loading 2, loading...
loaded 2
loaded 1
loaded 0
checking if components are loaded
component 2 loaded
component 1 loaded
adding to data store: {2: 'loaded', 1: 'loaded'}
checking if components are loaded
component 0 loaded
adding to data store: {0: 'loaded'}
loading_data={'0': 'loaded', '1': 'loaded', '2': 'loaded'}
loading 5, loading...
loading 4, loading...
checking if components are loaded
adding to data store: {}
loading 3, loading...
loading_data={'0': 'loaded', '1': 'loaded', '2': 'loaded'}
loaded 5
loaded 4
loaded 3
checking if components are loaded
component 5 loaded
adding to data store: {5: 'loaded'}
checking if components are loaded
component 4 loaded
adding to data store: {4: 'loaded'}
checking if components are loaded
component 3 loaded
adding to data store: {3: 'loaded'}
loading_data={'0': 'loaded', '1': 'loaded', '2': 'loaded', '3': 'loaded', '4': 'loaded', '5': 'loaded'}
...
loading_data={'0': 'loaded', '1': 'loaded', '2': 'loaded', '3': 'loaded', '4': 'loaded', '5': 'loaded', ... '20': 'loaded'}
FULLY LOADED

Exhibited Behaviour

After all components are loaded, the bottom text does not update to say "FULLY LOADED" and we see that the "loading_data" dictionary has not received all of the updates that were sent to it, as it does not include every index from 0 to 20.

loading_data=None
loading 2, loading...
loading 1, loading...
checking if components are loaded
adding to data store: {}
loading 0, loading...
loading_data={}
loaded 1
loaded 0
loaded 2
checking if components are loaded
component 0 loaded
adding to data store: {0: 'loaded'}
checking if components are loaded
component 1 loaded
adding to data store: {1: 'loaded'}
checking if components are loaded
component 2 loaded
adding to data store: {2: 'loaded'}
loading_data={'2': 'loaded'}
loading 5, loading...
loading 4, loading...
checking if components are loaded
adding to data store: {}
loading 3, loading...
loading_data={'2': 'loaded'}
loaded 5
loaded 4
loaded 3
checking if components are loaded
component 5 loaded
adding to data store: {5: 'loaded'}
checking if components are loaded
component 4 loaded
component 3 loaded
adding to data store: {4: 'loaded', 3: 'loaded'}
loading_data={'2': 'loaded', '3': 'loaded', '4': 'loaded'}
loading 8, loading...
loading 7, loading...
checking if components are loaded
adding to data store: {}
loading 6, loading...
loading_data={'2': 'loaded', '3': 'loaded', '4': 'loaded'}
loaded 8
loaded 6
loaded 7
checking if components are loaded
component 8 loaded
adding to data store: {8: 'loaded'}
loading_data={'2': 'loaded', '3': 'loaded', '4': 'loaded', '8': 'loaded'}
checking if components are loaded
component 7 loaded
component 6 loaded
adding to data store: {7: 'loaded', 6: 'loaded'}
loading_data={'2': 'loaded', '3': 'loaded', '4': 'loaded', '6': 'loaded', '7': 'loaded', '8': 'loaded'}
loading 11, loading...
loading 10, loading...
checking if components are loaded
adding to data store: {}
loading 9, loading...
loading_data={'2': 'loaded', '3': 'loaded', '4': 'loaded', '6': 'loaded', '7': 'loaded', '8': 'loaded'}
loaded 11
loaded 9
loaded 10
checking if components are loaded
component 11 loaded
adding to data store: {11: 'loaded'}
checking if components are loaded
component 9 loaded
component 10 loaded
adding to data store: {9: 'loaded', 10: 'loaded'}
loading_data={'2': 'loaded', '3': 'loaded', '4': 'loaded', '6': 'loaded', '7': 'loaded', '8': 'loaded', '9': 'loaded', '10': 'loaded'}
loading 14, loading...
loading 13, loading...
checking if components are loaded
adding to data store: {}
loading 12, loading...
loading_data={'2': 'loaded', '3': 'loaded', '4': 'loaded', '6': 'loaded', '7': 'loaded', '8': 'loaded', '9': 'loaded', '10': 'loaded'}
loaded 14
loaded 12
loaded 13
checking if components are loaded
component 14 loaded
adding to data store: {14: 'loaded'}
checking if components are loaded
component 13 loaded
component 12 loaded
adding to data store: {13: 'loaded', 12: 'loaded'}
loading_data={'2': 'loaded', '3': 'loaded', '4': 'loaded', '6': 'loaded', '7': 'loaded', '8': 'loaded', '9': 'loaded', '10': 'loaded', '12': 'loaded', '13': 'loaded'}
loading 17, loading...
loading 16, loading...
checking if components are loaded
adding to data store: {}
loading 15, loading...
loading_data={'2': 'loaded', '3': 'loaded', '4': 'loaded', '6': 'loaded', '7': 'loaded', '8': 'loaded', '9': 'loaded', '10': 'loaded', '12': 'loaded', '13': 'loaded'}
loaded 17
loaded 16
loaded 15
checking if components are loaded
component 17 loaded
adding to data store: {17: 'loaded'}
checking if components are loaded
component 16 loaded
component 15 loaded
adding to data store: {16: 'loaded', 15: 'loaded'}
loading_data={'2': 'loaded', '3': 'loaded', '4': 'loaded', '6': 'loaded', '7': 'loaded', '8': 'loaded', '9': 'loaded', '10': 'loaded', '12': 'loaded', '13': 'loaded', '15': 'loaded', '16': 'loaded'}
loading 20, loading...
loading 19, loading...
checking if components are loaded
adding to data store: {}
loading 18, loading...
loading_data={'2': 'loaded', '3': 'loaded', '4': 'loaded', '6': 'loaded', '7': 'loaded', '8': 'loaded', '9': 'loaded', '10': 'loaded', '12': 'loaded', '13': 'loaded', '15': 'loaded', '16': 'loaded'}
loaded 20
loaded 19
loaded 18
checking if components are loaded
component 18 loaded
adding to data store: {18: 'loaded'}
checking if components are loaded
component 19 loaded
component 20 loaded
adding to data store: {19: 'loaded', 20: 'loaded'}
loading_data={'2': 'loaded', '3': 'loaded', '4': 'loaded', '6': 'loaded', '7': 'loaded', '8': 'loaded', '9': 'loaded', '10': 'loaded', '12': 'loaded', '13': 'loaded', '15': 'loaded', '16': 'loaded', '19': 'loaded', '20': 'loaded'}