Skip to content

pymdea.core

Diffusion entropy analysis core methods.

DeaEngine(loader, hist_bins='doane', windows=250, window_stop=0.25)

Run diffusion entropy analysis.

Parameters:

  • loader

    (DeaLoader) –

    An instance of the DeaLoader class containing data to be analysed.

  • hist_bins

    (int | {auto, fd, doane, scott, stone, rice, sturges}, default: 'doane' ) –

    Number of bins, or method by which to calculate it, to use for the histogram in the Shannon entropy calculation. Refer to numpy.histogram_bin_edges for details about the binning methods.

  • window_stop

    (float, default: 0.25 ) –

    Proportion of data length at which to cap window length. For example, if set to 0.25, 0.25 * len(data) will be the maximum window length. Must be a float in (0, 1].

  • windows

    (int, default: 250 ) –

    Number of window lengths to use and fit over. Window lengths will be evenly spaced in log-scale.

Source code in src/pymdea/core.py 
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
def __init__(
    self: Self,
    loader: DeaLoader,
    hist_bins: int
    | Literal["fd", "doane", "scott", "stone", "rice", "sturges"] = "doane",
    windows: int = 250,
    window_stop: float = 0.25,
) -> Self:
    """Run diffusion entropy analysis.

    Parameters
    ----------
    loader : DeaLoader
        An instance of the DeaLoader class containing data to be
        analysed.
    hist_bins : int | {"auto", "fd", "doane", "scott", "stone", "rice", "sturges"}
        Number of bins, or method by which to calculate it, to use
        for the histogram in the Shannon entropy calculation. Refer
        to `numpy.histogram_bin_edges` for details about the
        binning methods.
    window_stop : float
        Proportion of data length at which to cap window length.
        For example, if set to 0.25, 0.25 * len(data) will be the
        maximum window length. Must be a float in (0, 1].
    windows : int
        Number of window lengths to use and fit over.
        Window lengths will be evenly spaced in log-scale.

    """
    console = Console()
    logging.basicConfig(
        level="INFO",
        format="%(funcName)s: %(message)s",
        datefmt="%Y-%m-%d %H:%M:%S",
        handlers=[RichHandler(console=console)],
    )
    self.log = logging.getLogger("rich")
    if window_stop <= 0 or window_stop > 1:
        msg = f"Parameter 'window_stop' must be in (0, 1], got: {window_stop}"
        raise ValueError(msg)
    n_windows = int(np.floor(window_stop * len(loader.data)))
    if windows > n_windows:
        msg = (
            f"Parameter max_fit={windows} longer than "
            f"window_stop * len(data) = {n_windows}, "
            f"{n_windows} will be used"
        )
        self.log.warning(msg)
    self.data = loader.data
    self.hist_bins = hist_bins
    self.window_stop = window_stop
    self.max_fit = windows
    self.progress = Progress(
        SpinnerColumn(),
        TextColumn("[progress.percentage]{task.percentage:>3.0f}%"),
        BarColumn(),
        TextColumn("eta"),
        TimeRemainingColumn(),
        TextColumn("elapsed"),
        TimeElapsedColumn(),
        console=console,
    )

analyze_with_stripes(fit_start, fit_stop, fit_method='siegel', stripes=20)

Run a modified diffusion entropy analysis.

Parameters:

  • fit_start

    (float) –

    Fraction of maximum window length at which to start linear fit.

  • fit_stop

    (float) –

    Fraction of maximum window length at which to stop linear fit.

  • fit_method

    (str {"siegel", "theilsen", "ls"}, default: 'siegel' ) –

    Linear fit method to use. By default "siegel"

  • stripes

    (int, default: 20 ) –

    Number of stripes to apply to input time-series during analysis.

Returns:

  • Self @ Engine –

    Object containing the results and inputs of the diffusion entropy analysis.

Raises:

  • ValueError –

    If n_stripes < 2. At least two stripes must be applied for DEA to provide a meaningful result.

Notes

Prefer the siegel or theilsen methods. Least squares linear fits can introduce bias when done over log-scale data, see Clauset, A., Shalizi, C.R. and Newman, M.E., 2009. Power-law distributions in empirical data. SIAM review, 51(4), pp.661-703. https://doi.org/10.1137/070710111. https://arxiv.org/pdf/0706.1062.pdf.

Source code in src/pymdea/core.py 
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
def analyze_with_stripes(
    self: Self,
    fit_start: int,
    fit_stop: int,
    fit_method: Literal["siegel", "theilsen", "ls"] = "siegel",
    stripes: int = 20,
) -> Self:
    """Run a modified diffusion entropy analysis.

    Parameters
    ----------
    fit_start : float
        Fraction of maximum window length at which to start linear fit.
    fit_stop : float
        Fraction of maximum window length at which to stop linear fit.
    fit_method : str {"siegel", "theilsen", "ls"}, optional
        Linear fit method to use. By default "siegel"
    stripes : int, optional, default: 20
        Number of stripes to apply to input time-series during
        analysis.

    Returns
    -------
    Self @ Engine
        Object containing the results and inputs of the diffusion
        entropy analysis.

    Raises
    ------
    ValueError
        If n_stripes < 2. At least two stripes must be applied for
        DEA to provide a meaningful result.

    Notes
    -----
    Prefer the siegel or theilsen methods. Least squares linear
    fits can introduce bias when done over log-scale data, see
    Clauset, A., Shalizi, C.R. and Newman, M.E., 2009. Power-law
    distributions in empirical data. SIAM review, 51(4), pp.661-703.
    https://doi.org/10.1137/070710111.
    https://arxiv.org/pdf/0706.1062.pdf.

    """
    if stripes < 2:  # noqa: PLR2004
        msg = "n_stripes must be greater than 1"
        raise ValueError(msg)
    self.number_of_stripes = stripes
    self.fit_start = fit_start
    self.fit_stop = fit_stop
    self.fit_method = fit_method
    self._apply_stripes()
    self._get_events()
    self._make_trajectory()
    self._calculate_entropy()
    self._calculate_scaling()
    self._calculate_mu()
    self.print_result()

analyze_without_stripes(fit_start, fit_stop, fit_method='siegel')

Run a regular diffusion entropy analysis.

Parameters:

  • fit_start

    (float) –

    Fraction of maximum window length at which to start linear fit.

  • fit_stop

    (float) –

    Fraction of maximum window length at which to stop linear fit.

  • fit_method

    (str {"siegel", "theilsen", "ls"}, default: 'siegel' ) –

    Linear fit method to use. By default "siegel"

Returns:

  • Self @ Engine –

    Object containing the results and inputs of the diffusion entropy analysis.

Notes

Prefer the siegel or theilsen methods. Least squares linear fits can introduce bias when done over log-scale data, see Clauset, A., Shalizi, C.R. and Newman, M.E., 2009. Power-law distributions in empirical data. SIAM review, 51(4), pp.661-703. https://doi.org/10.1137/070710111. https://arxiv.org/pdf/0706.1062.pdf.

Source code in src/pymdea/core.py 
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
def analyze_without_stripes(
    self: Self,
    fit_start: int,
    fit_stop: int,
    fit_method: Literal["siegel", "theilsen", "ls"] = "siegel",
) -> Self:
    """Run a regular diffusion entropy analysis.

    Parameters
    ----------
    fit_start : float
        Fraction of maximum window length at which to start linear fit.
    fit_stop : float
        Fraction of maximum window length at which to stop linear fit.
    fit_method : str {"siegel", "theilsen", "ls"}, optional
        Linear fit method to use. By default "siegel"

    Returns
    -------
    Self @ Engine
        Object containing the results and inputs of the diffusion
        entropy analysis.

    Notes
    -----
    Prefer the siegel or theilsen methods. Least squares linear
    fits can introduce bias when done over log-scale data, see
    Clauset, A., Shalizi, C.R. and Newman, M.E., 2009. Power-law
    distributions in empirical data. SIAM review, 51(4), pp.661-703.
    https://doi.org/10.1137/070710111.
    https://arxiv.org/pdf/0706.1062.pdf.

    """
    self.trajectory = self.data
    self.fit_start = fit_start
    self.fit_stop = fit_stop
    self.fit_method = fit_method
    self._calculate_entropy()
    self._calculate_scaling()
    self._calculate_mu()
    self.print_result()

print_result()

Print out result of analysis.

Source code in src/pymdea/core.py 
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
def print_result(self: Self) -> str:
    """Print out result of analysis."""
    self.result = Table(box=box.SIMPLE)
    self.result.add_column("δ")
    self.result.add_column("μ (rule 1)")
    self.result.add_column("μ (rule 2)")
    self.result.add_column("μ (rule 3)")
    self.result.add_row(
        f"{self.delta:.5f}",
        f"{self.mu1:.5f}",
        f"{self.mu2:.5f}",
        f"{self.mu3:.5f}",
    )
    console = Console()
    console.print(self.result)
    return self

DeaLoader()

Load data for a diffusion entropy analysis.

Source code in src/pymdea/core.py 
35
36
def __init__(self: Self) -> Self:
    """Load data for a diffusion entropy analysis."""

make_diffusion_process(kind='gn', length=10000, a=0)

Generate diffusion process data.

Parameters:

  • kind

    (str {"cn", "gn", "fgn", "fbm"}, default: "cn" ) –

    Type of diffusion noise to generate. If "cn", generate a colored noise with spectral power a. If "gn", generate a Gaussian noise. If "fgn", generate a fractional Gaussian noise with Hurst index H = a. If "fbm", generate a fractional Brownian motion with Hurst index H=a.

  • length

    (int, default: 10000 ) –

    Length of time-series to generate.

  • a

    (float, default: 0 ) –

    Only used if kind is "fgn", "fbm", or "cn". If kind is "fgn" or "fbm", this sets the Hurst index of the process. If kind is "cn" this sets the index of the power law spectrum for the noise, 1/(f^a).

Returns:

  • Self @ Loader –

    An instance of the Loader object.

Source code in src/pymdea/core.py 
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
def make_diffusion_process(
    self: Self,
    kind: Literal["cn", "gn", "fgn", "fbm"] = "gn",
    length: int = 10000,
    a: float = 0,
) -> Self:
    """Generate diffusion process data.

    Parameters
    ----------
    kind : str {"cn", "gn", "fgn", "fbm"}, optional, default "cn"
        Type of diffusion noise to generate. If "cn", generate a
        colored noise with spectral power `a`. If "gn", generate a
        Gaussian noise. If "fgn", generate a fractional Gaussian
        noise with Hurst index H = `a`. If "fbm", generate a
        fractional Brownian motion with Hurst index H=`a`.
    length : int, optional, default 10000
        Length of time-series to generate.
    a : float, optional, default 0
        Only used if `kind` is "fgn", "fbm", or "cn". If `kind` is
        "fgn" or "fbm", this sets the Hurst index of the process.
        If `kind` is "cn" this sets the index of the power law
        spectrum for the noise, 1/(f^`a`).

    Returns
    -------
    Self @ Loader
        An instance of the Loader object.

    """
    if kind == "gn":
        process = stochastic.processes.noise.GaussianNoise()
        self.data = process.sample(length)
    if kind == "cn":
        process = stochastic.processes.noise.ColoredNoise(beta=a)
        self.data = process.sample(length)
    if kind == "fgn":
        process = stochastic.processes.noise.FractionalGaussianNoise(hurst=a)
        self.data = process.sample(length)
    if kind == "fbm":
        process = stochastic.processes.continuous.FractionalBrownianMotion(hurst=a)
        self.data = process.sample(length)
    return self

make_sample_data(length=100000, seed=1)

Generate an array of sample data.

Parameters:

  • length

    (int, default: 100000 ) –

    Number of time-steps to produce in the sample data.

  • seed

    (int, default: 1 ) –

    Seed for random number generation.

Returns:

  • Self @ Loader –

    An instance of the Loader object.

Source code in src/pymdea/core.py 
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
def make_sample_data(self: Self, length: int = 100000, seed: int = 1) -> np.ndarray:
    """Generate an array of sample data.

    Parameters
    ----------
    length : int, optional, default: 100000
        Number of time-steps to produce in the sample data.
    seed : int, optional, default: 1
        Seed for random number generation.

    Returns
    -------
    Self @ Loader
        An instance of the Loader object.

    """
    rng = np.random.default_rng(seed=seed)
    random_walk = rng.choice([-1, 1], size=length).cumsum()
    random_walk[0] = 0  # always start from 0
    self.seed = seed
    self.data = random_walk
    return self

read_data_file(filepath, column_name)

Read input data from file.

Parameters:

  • filepath

    (str) –

    System path to a file containing data. Must include the full file name, including the extension. Example: "/example/path/to/file.csv"

  • column_name

    (str) –

    Name of the column in the data file which contains the time series data values.

Returns:

  • Self @ Loader –

    An instance of the Loader object.

Raises:

  • ValueError –

    If filepath points to a file of type other than CSV. Support for more types of files is a work in progress.

Source code in src/pymdea/core.py 
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
def read_data_file(self: Self, filepath: str, column_name: str) -> pl.DataFrame:
    """Read input data from file.

    Parameters
    ----------
    filepath : str
        System path to a file containing data. Must include the
        full file name, including the extension. Example:
        "/example/path/to/file.csv"
    column_name : str
        Name of the column in the data file which contains the time
        series data values.

    Returns
    -------
    Self @ Loader
        An instance of the Loader object.

    Raises
    ------
    ValueError
        If filepath points to a file of type other than
        CSV. Support for more types of files is a work in
        progress.

    """
    filepath = Path(filepath)
    supported_types = [".csv"]
    if filepath.suffix not in supported_types:
        msg = f"filetype must be one of: {supported_types}."
        raise ValueError(msg)
    if filepath.suffix == ".csv":
        self.data = pl.scan_csv(filepath).select(column_name).to_numpy()
    return self