batched_indices
- sisl.utils.batched_indices(ref, y, axis=-1, atol=1e-8, batch_size=200, diff_func=None)[source]
Locate x in ref by examining
np.abs(diff_func(ref - y)) <= atol
This method is necessary for very large groups of data since the
ref-y
calls will use broad-casting to create very large memory chunks.This method will allow broad-casting arrays up to a size of batch_size.
The memory is calculating using
np.prod(ref.shape) / ref.shape[axis] * n
such thatn
chunks of y is using the batch_size MB of memory.n
will minimally be 1, regardless of batch_size.- Parameters:
ref (
array_like
) – reference array where we wish to locate the indices of y iny (
array_like
of1D
or2D
) – array to locate in ref. For 2D arrays and axis not None,axis (
int
orNone
, optional) – which axis to do a logical reduction along, if None it means that they are 1D arrays and no axis will be reduced, i.e. same asref.ravel() - y.reshape(-1, 1)
but retaining indices of the same dimension as ref.atol (
float
orarray_like
, optional) – absolute tolerance for comparing values, for array_like values it must have the same length asref.shape[axis]
batch_size (
float
, optional) – maximum size of broad-casted array. Internal algorithms will determine the chunk size of ydiff_func (
callable
, optional) – function to post-process the difference values, defaults to do nothing. Should have an interfacedef diff_func(diff)
- Returns:
tuple
ofndarray
– the indices for each ref dimension that matches the values in y. the returned indices behave likenumpy.nonzero
.