Operations on COO
arrays¶
Operators¶
COO
objects support a number of operations. They interact with scalars,
Numpy arrays, other COO
objects, and
scipy.sparse.spmatrix
objects, all following standard Python and Numpy
conventions.
For example, the following Numpy expression produces equivalent results for both Numpy arrays, COO arrays, or a mix of the two:
np.log(X.dot(beta.T) + 1)
However some operations are not supported, like inplace operations, operations that implicitly cause dense structures, or numpy functions that are not yet implemented for sparse arrays
x += y # inplace operations not supported
x + 1 # operations that produce dense results not supported
np.svd(x) # sparse svd not implemented
This page describes those valid operations, and their limitations.
elemwise
¶
This function allows you to apply any arbitrary broadcasting function to any number of arguments
where the arguments can be SparseArray
objects or scipy.sparse.spmatrix
objects.
For example, the following will add two arrays:
sparse.elemwise(np.add, x, y)
Auto-Densification¶
Operations that would result in dense matrices, such as binary
operations with Numpy arrays objects or certain operations with
scalars are not allowed and will raise a ValueError
. For example,
all of the following will raise a ValueError
. Here, x
and
y
are COO
objects.
x == y
x + 5
x == 0
x != 5
x / y
However, all of the following are valid operations.
x + 0
x != y
x + y
x == 5
5 * x
x / 7.3
x != 0
If densification is needed, it must be explicit. In other words, you must call
COO.todense
on the COO
object. If both operands are COO
,
both must be densified.
Warning
Previously, operations with Numpy arrays were sometimes supported. Now,
it is necessary to convert Numpy arrays to COO
objects.
Operations with scipy.sparse.spmatrix
¶
Certain operations with scipy.sparse.spmatrix
are also supported.
For example, the following are all allowed if y
is a scipy.sparse.spmatrix
:
x + y
x - y
x * y
x > y
x < y
In general, if operating on a scipy.sparse.spmatrix
is the same as operating
on COO
, as long as it is to the right of the operator.
Note
Results are not guaranteed if x
is a scipy.sparse.spmatrix
.
For this reason, we recommend that all Scipy sparse matrices should be explicitly
converted to COO
before any operations.
Broadcasting¶
All binary operators support broadcasting.
This means that (under certain conditions) you can perform binary operations
on arrays with unequal shape. Namely, when the shape is missing a dimension,
or when a dimension is 1
. For example, performing a binary operation
on two COO
arrays with shapes (4,)
and (5, 1)
yields
an object of shape (5, 4)
. The same happens with arrays of shape
(1, 4)
and (5, 1)
. However, (4, 1)
and (5, 1)
will raise a ValueError
.
Full List of Operators¶
Here, x
and y
can be COO
arrays,
numpy.ndarray
objects or scalars, keeping in mind auto
densification rules. In addition, y
can also
be a scipy.sparse.spmatrix
The following operators are supported:
Basic algebraic operations
operator.add
(x + y
)operator.neg
(-x
)operator.sub
(x - y
)operator.mul
(x * y
)operator.truediv
(x / y
)operator.floordiv
(x // y
)operator.pow
(x ** y
)
Comparison operators
operator.eq
(x == y
)operator.ne
(x != y
)operator.gt
(x > y
)operator.ge
(x >= y
)operator.lt
(x < y
)operator.le
(x <= y
)
Bitwise operators
operator.and_
(x & y
)operator.or_
(x | y
)operator.xor
(x ^ y
)
Bit-shifting operators
operator.lshift
(x << y
)operator.rshift
(x >> y
)
Note
In-place operators are not supported at this time.
Element-wise Operations¶
COO
arrays support a variety of element-wise operations. However, as
with operators, operations that map zero to a nonzero value are not supported.
To illustrate, the following are all possible, and will produce another
COO
array:
np.abs(x)
np.sin(x)
np.sqrt(x)
np.conj(x)
np.expm1(x)
np.log1p(x)
However, the following are all unsupported and will raise a ValueError
:
np.exp(x)
np.cos(x)
np.log(x)
Notice that you can apply any unary or binary numpy.ufunc to COO
arrays, and numpy.ndarray
objects and scalars and it will work so
long as the result is not dense. When applying to numpy.ndarray
objects,
we check that operating on the array with zero would always produce a zero.
Reductions¶
COO
objects support a number of reductions. However, not all important
reductions are currently implemented (help welcome!) All of the following
currently work:
x.sum(axis=1)
np.max(x)
np.min(x, axis=(0, 2))
x.prod()
Note
If you are performing multiple reductions along the same axes, it may
be beneficial to call COO.enable_caching
.
COO.reduce
¶
This method can take an arbitrary numpy.ufunc and performs a reduction using that method. For example, the following will perform a sum:
x.reduce(np.add, axis=1)
Note
This library currently performs reductions by grouping together all coordinates along the supplied axes and reducing those. Then, if the number in a group is deficient, it reduces an extra time with zero. As a result, if reductions can change by adding multiple zeros to it, this method won’t be accurate. However, it works in most cases.
Indexing¶
COO
arrays can be indexed
just like regular
numpy.ndarray
objects. They support integer, slice and boolean indexing.
However, currently, numpy advanced indexing is not properly supported. This
means that all of the following work like in Numpy, except that they will produce
COO
arrays rather than numpy.ndarray
objects, and will produce
scalars where expected. Assume that z.shape
is (5, 6, 7)
z[0]
z[1, 3]
z[1, 4, 3]
z[:3, :2, 3]
z[::-1, 1, 3]
z[-1]
z[[True, False, True, False, True], 3, 4]
All of the following will raise an IndexError
, like in Numpy 1.13 and later.
z[6]
z[3, 6]
z[1, 4, 8]
z[-6]
z[[True, True, False, True], 3, 4]
Note
Numpy advanced indexing is currently not supported.
Other Operations¶
COO
arrays support a number of other common operations. Among them are
dot
, tensordot
, concatenate
and stack
, transpose
and reshape
.
You can view the full list on the API reference page.