9.1. Indexing

Multidimensional indices enable the user to define on how to replicate transformations and variables. The main operations for indices are:

  1. Iterate over all possible combinations of the values of the indices.

  2. Get the current values of indices.

  3. Format strings using current values of indices.

  4. Get a subset of indices.

  5. Merge few subsets of indices.

9.1.1. 0d case

Let us start from the simplest possible example of empty index:

01_indexing.py
 1from gna.expression.index import *
 2
 3#
 4# 0d index
 5#
 6nidx = NIndex(fromlist=[])
 7
 8print('Test 0d index')
 9for i, nit in enumerate(nidx):
10    print('  iteration', i)
11    print('    index:    ', nit.current_format() or '<empty string>')
12    print('    full name:', nit.current_format(name='var'))
13    print('    values:   ', nit.current_values())
14    print()

The index is created by instantiating an NIndex class:

nidx = NIndex(fromlist=[])

The NIndex instance is ready for iteration:

for i, nit in enumerate(nidx):
    print('  iteration', i)
    print('    index:    ', nit.current_format() or '<empty string>')
    print('    full name:', nit.current_format(name='var'))
    print('    values:   ', nit.current_values())

A copy of NIndex is returned on each iteration with current values set. A current_values() method returns a tuple with all current index values, which is empty in our case. A current_format() method formats a string using the current values. If a name is provided as an argument it is also added to the format.

The following output is produced. As one may see, empty index still produces a single iteration.

Test 0d index
  iteration 0
    index:     <empty string>
    full name: var
    values:    ()

9.1.2. 1d case

Now let us create an 1d index: to do this we add a definition of one of the indices:

1nidx = NIndex(fromlist=[
2    ('i', 'index', ['1', '2', '3'])
3    ])

Typical definition contains three items: short name, long name and a list with values. Short name will be used for reference. Long name is used for string formatting.

When iterated,

1for i, nit in enumerate(nidx):
2    print('  iteration', i, end=': ')
3    print('    values:   ', nit.current_values())

the resulting index will contain the value from the list on each iteration:

iteration 0:     values:    ('1',)
iteration 1:     values:    ('2',)
iteration 2:     values:    ('3',)

The current_format() returns a single string for each iteration.

1for i, nit in enumerate(nidx):
2    print('  iteration', i, end=': ')
3    print('    index:    ', nit.current_format())

For 1d case the string will be equal to each of the values:

iteration 0:     index:     1
iteration 1:     index:     2
iteration 2:     index:     3

When name field is added to the current_format() call, name is joined to the values of indices with . as a separator:

1for i, nit in enumerate(nidx):
2    print('  iteration', i, end=': ')
3    print('    full name:', nit.current_format(name='var'))

iteration 0:     full name: var.1
iteration 1:     full name: var.2
iteration 2:     full name: var.3

9.1.3. 2d case

In order to make 2d multi index just add another index specification:

1nidx = NIndex(fromlist=[
2    ('i', 'index', ['1', '2', '3']),
3    ('j', 'element', ['a', 'b'])
4    ])

The iteration of the 2d index

1for i, nit in enumerate(nidx):
2    print('  iteration', i)
3    print('    index:    ', nit.current_format())
4    print('    full name:', nit.current_format(name='var'))
5    print('    values:   ', nit.current_values())

will produce all possible pairs of the values of i and j indices:

iteration 0
  index:     1.a
  full name: var.1.a
  values:    ('1', 'a')

iteration 1
  index:     1.b
  full name: var.1.b
  values:    ('1', 'b')

iteration 2
  index:     2.a
  full name: var.2.a
  values:    ('2', 'a')

iteration 3
  index:     2.b
  full name: var.2.b
  values:    ('2', 'b')

iteration 4
  index:     3.a
  full name: var.3.a
  values:    ('3', 'a')

iteration 5
  index:     3.b
  full name: var.3.b
  values:    ('3', 'b')

The order of the indices is kept to the order of the initialization.

9.1.4. 3d case and name position

For 3d case we will introduce the indices z, s and d. Where s is used for sources, d is used for detectors. Consider the case when we want to create several clones of the same model, the z will be used to index them. In this case we also would like so that the variables and outputs of the same models were stored together. This may be achieved by putting the name after the clone index:

1nidx = NIndex(fromlist=[
2    ('z', 'clone', ['clone_00', 'clone_01']),
3    'name',
4    ('s', 'source', ['SA', 'SB']),
5    ('d', 'detector', ['D1', 'D2'])
6    ])

When the string name is passed instead of index definition, the position of the name will be used to position the actual name.

Now iterating nidx will produce all possible combinations of the z, s and d indices in the order they were specified:

iteration 0
  full name: clone_00.var.SA.D1
  values:    ('clone_00', 'var', 'SA', 'D1')

iteration 1
  full name: clone_00.var.SA.D2
  values:    ('clone_00', 'var', 'SA', 'D2')

iteration 2
  full name: clone_00.var.SB.D1
  values:    ('clone_00', 'var', 'SB', 'D1')

iteration 3
  full name: clone_00.var.SB.D2
  values:    ('clone_00', 'var', 'SB', 'D2')

iteration 4
  full name: clone_01.var.SA.D1
  values:    ('clone_01', 'var', 'SA', 'D1')

iteration 5
  full name: clone_01.var.SA.D2
  values:    ('clone_01', 'var', 'SA', 'D2')

iteration 6
  full name: clone_01.var.SB.D1
  values:    ('clone_01', 'var', 'SB', 'D1')

iteration 7
  full name: clone_01.var.SB.D2
  values:    ('clone_01', 'var', 'SB', 'D2')

Also, note that the name in this example goes after the clone index.

9.1.5. 4d indexing and partial iteration

Let us now look at 4d case.

1nidx = NIndex(fromlist=[
2    ('z', 'clone', ['clone_00', 'clone_01']),
3    'name',
4    ('s', 'source', ['SA', 'SB']),
5    ('d', 'detector', ['D1', 'D2']),
6    ('e', 'element', ['e1', 'e2', 'e3'])
7    ])

It is often needed to iterate only a part of the indices of a multidimensional index. This may be achieved by split(short_names) method:

1nidx_major, nidx_minor=nidx.split(('s', 'd'))

It produces a pair of NIndex instances. First one contains the indices from the list of short names, passed as argument. In this case it contains s and d. The second NIndex contains all the other indices: z and e in this case.

These two instances may be iterated independently:

1for i_major, nit_major in enumerate(nidx_major):
2    print('  major iteration', i_major)
3    print('    major values:   ', nit_major.current_values())
4
5    for j_minor, nit_minor in enumerate(nidx_minor):
6        print('      minor iteration', j_minor)
7        print('        minor values:  ', nit_minor.current_values())

This kind of iteration is useful, for example, in the cases, when variables depend only on a part of the indices.

The partially iterated indices may be combined together:

1        nit = nit_major + nit_minor
2        print('        full name:     ', nit.current_format(name='var'))

The current_format() method has an optional argument: python format specification. In this case the string will be formatted using the values of the current indices, which may be referenced by both short and long names:

1        print('        custom label:  ', nit.current_format('Flux from {source} to {detector} element {element} ({clone})'))

The format string may reference any other positional and keyword arguments which should be passed to the current_format() as well.

The output for the example with 4d index is the following:

major iteration 0
  major values:    ('SA', 'D1')
    minor iteration 0
      minor values:   ('clone_00', 'e1')
      full name:      clone_00.var.SA.D1.e1
      custom label:   Flux from SA to D1 element e1 (clone_00)
    minor iteration 1
      minor values:   ('clone_00', 'e2')
      full name:      clone_00.var.SA.D1.e2
      custom label:   Flux from SA to D1 element e2 (clone_00)
    minor iteration 2
      minor values:   ('clone_00', 'e3')
      full name:      clone_00.var.SA.D1.e3
      custom label:   Flux from SA to D1 element e3 (clone_00)
    minor iteration 3
      minor values:   ('clone_01', 'e1')
      full name:      clone_01.var.SA.D1.e1
      custom label:   Flux from SA to D1 element e1 (clone_01)
    minor iteration 4
      minor values:   ('clone_01', 'e2')
      full name:      clone_01.var.SA.D1.e2
      custom label:   Flux from SA to D1 element e2 (clone_01)
    minor iteration 5
      minor values:   ('clone_01', 'e3')
      full name:      clone_01.var.SA.D1.e3
      custom label:   Flux from SA to D1 element e3 (clone_01)

We stop after the first major iteration to keep the output short.

9.1.6. Dependant indices

Sometimes it is useful to have several indices for the same thing. For example, imagine the detector elements come in groups and may be referenced by both element number or group number. The group index then may be used to specify some common variables for the group.

This situation may be handled by the NIndex as well. The resolution rules are the following:

  1. The group indices are called slave indices, while the elements indices within groups are called master indices.

  2. When we work with a set of indices, that contains a slave index but not its master, the slave index values will be iterated.

  3. In any case when slave index is combined with its master, only master index will be iterated. In the same time the relevant slave index value may be obtained for each master value on each iteration.

To work with dependant indices we need to tell the master index how its values are grouped:

1nidx = NIndex(fromlist=[
2    ('d', 'detector', ['D1', 'D2']),
3    ('s', 'source', ['SA', 'SB']),
4    ('g', 'group', ['g1', 'g2']),
5    ('e', 'element', ['e1', 'e2', 'e3'], dict(short='g', name='group', map=[('g1', ('e1', 'e2')), ('g2', ('e3',)) ]))
6    ])

In our case for e index we have provided a dictionary with short and long names of the slave index, as well as the list of pairs slave index and master indices. In our example group g1 contains elements e1 and e2, the group g2 contains the only element e3.

Now let us look how this work. Let us split the multidimensional index in two:

1nidx_major, nidx_minor=nidx.split(('d', 'g'))
2nidx_e=nidx.get_subset('e')

The first group contains indices d and g, while the second contains the index s. Index e is not used since g was required. We have also requested multi-index with e index for later use. The iteration over nidx_major and nidx_minor produces the following output:

major iteration 0
  major values:    ('D1', 'g1')
    full values 0: ('D1', 'SA', 'g1')
    full values 1: ('D1', 'SB', 'g1')

major iteration 1
  major values:    ('D1', 'g2')
    full values 0: ('D1', 'SA', 'g2')
    full values 1: ('D1', 'SB', 'g2')

major iteration 2
  major values:    ('D2', 'g1')
    full values 0: ('D2', 'SA', 'g1')
    full values 1: ('D2', 'SB', 'g1')

major iteration 3
  major values:    ('D2', 'g2')
    full values 0: ('D2', 'SA', 'g2')
    full values 1: ('D2', 'SB', 'g2')

As one may see, there is each combination of d, s and g values.

Now let us combine nidx_major with nidx_e. The former one contains index g while the latter one contains index e, which may not be combined together.

1nidx_major+=nidx_e

The result will contain only index e, but the relevant g value may be obtained by specifying the format string:

1        print('      formatted string %i:'%j_minor, nit.current_format('Element {element} in group {group}'))

Here is the output:

major iteration 0
  major values:    ('D1', 'e1')
    full values 0:      ('D1', 'SA', 'e1')
    formatted string 0: Element e1 in group g1
    full values 1:      ('D1', 'SB', 'e1')
    formatted string 1: Element e1 in group g1

major iteration 1
  major values:    ('D1', 'e2')
    full values 0:      ('D1', 'SA', 'e2')
    formatted string 0: Element e2 in group g1
    full values 1:      ('D1', 'SB', 'e2')
    formatted string 1: Element e2 in group g1

major iteration 2
  major values:    ('D1', 'e3')
    full values 0:      ('D1', 'SA', 'e3')
    formatted string 0: Element e3 in group g2
    full values 1:      ('D1', 'SB', 'e3')
    formatted string 1: Element e3 in group g2

major iteration 3
  major values:    ('D2', 'e1')
    full values 0:      ('D2', 'SA', 'e1')
    formatted string 0: Element e1 in group g1
    full values 1:      ('D2', 'SB', 'e1')
    formatted string 1: Element e1 in group g1

major iteration 4
  major values:    ('D2', 'e2')
    full values 0:      ('D2', 'SA', 'e2')
    formatted string 0: Element e2 in group g1
    full values 1:      ('D2', 'SB', 'e2')
    formatted string 1: Element e2 in group g1

major iteration 5
  major values:    ('D2', 'e3')
    full values 0:      ('D2', 'SA', 'e3')
    formatted string 0: Element e3 in group g2
    full values 1:      ('D2', 'SB', 'e3')
    formatted string 1: Element e3 in group g2

9.1.7. The full example

The complete code for this example may be found below:

01_indexing.py
  1from gna.expression.index import *
  2
  3#
  4# 0d index
  5#
  6nidx = NIndex(fromlist=[])
  7
  8print('Test 0d index')
  9for i, nit in enumerate(nidx):
 10    print('  iteration', i)
 11    print('    index:    ', nit.current_format() or '<empty string>')
 12    print('    full name:', nit.current_format(name='var'))
 13    print('    values:   ', nit.current_values())
 14    print()
 15
 16#
 17# 1d index
 18#
 19nidx = NIndex(fromlist=[
 20    ('i', 'index', ['1', '2', '3'])
 21    ])
 22
 23for i, nit in enumerate(nidx):
 24    print('  iteration', i, end=': ')
 25    print('    values:   ', nit.current_values())
 26print()
 27
 28
 29print('Test 1d index')
 30for i, nit in enumerate(nidx):
 31    print('  iteration', i, end=': ')
 32    print('    index:    ', nit.current_format())
 33print()
 34
 35for i, nit in enumerate(nidx):
 36    print('  iteration', i, end=': ')
 37    print('    full name:', nit.current_format(name='var'))
 38print()
 39
 40#
 41# 2d index
 42#
 43nidx = NIndex(fromlist=[
 44    ('i', 'index', ['1', '2', '3']),
 45    ('j', 'element', ['a', 'b'])
 46    ])
 47
 48print('Test 2d index')
 49for i, nit in enumerate(nidx):
 50    print('  iteration', i)
 51    print('    index:    ', nit.current_format())
 52    print('    full name:', nit.current_format(name='var'))
 53    print('    values:   ', nit.current_values())
 54
 55    print()
 56
 57#
 58# 3d index and arbitrary name position
 59#
 60nidx = NIndex(fromlist=[
 61    ('z', 'clone', ['clone_00', 'clone_01']),
 62    'name',
 63    ('s', 'source', ['SA', 'SB']),
 64    ('d', 'detector', ['D1', 'D2'])
 65    ])
 66print('Test 3d index and arbitrary name position')
 67for i, nit in enumerate(nidx):
 68    print('  iteration', i)
 69    print('    full name:', nit.current_format(name='var'))
 70    print('    values:   ', nit.current_values(name='var'))
 71
 72    print()
 73
 74#
 75# 4d index and separated iteration
 76#
 77nidx = NIndex(fromlist=[
 78    ('z', 'clone', ['clone_00', 'clone_01']),
 79    'name',
 80    ('s', 'source', ['SA', 'SB']),
 81    ('d', 'detector', ['D1', 'D2']),
 82    ('e', 'element', ['e1', 'e2', 'e3'])
 83    ])
 84print('Test 4d index and separated iteration')
 85nidx_major, nidx_minor=nidx.split(('s', 'd'))
 86for i_major, nit_major in enumerate(nidx_major):
 87    print('  major iteration', i_major)
 88    print('    major values:   ', nit_major.current_values())
 89
 90    for j_minor, nit_minor in enumerate(nidx_minor):
 91        print('      minor iteration', j_minor)
 92        print('        minor values:  ', nit_minor.current_values())
 93
 94        nit = nit_major + nit_minor
 95        print('        full name:     ', nit.current_format(name='var'))
 96        print('        custom label:  ', nit.current_format('Flux from {source} to {detector} element {element} ({clone})'))
 97
 98    print()
 99    break
100
101#
102# Dependant indices
103#
104nidx = NIndex(fromlist=[
105    ('d', 'detector', ['D1', 'D2']),
106    ('s', 'source', ['SA', 'SB']),
107    ('g', 'group', ['g1', 'g2']),
108    ('e', 'element', ['e1', 'e2', 'e3'], dict(short='g', name='group', map=[('g1', ('e1', 'e2')), ('g2', ('e3',)) ]))
109    ])
110
111print('Test 4d index and dependant indices')
112nidx_major, nidx_minor=nidx.split(('d', 'g'))
113nidx_e=nidx.get_subset('e')
114for i_major, nit_major in enumerate(nidx_major):
115    print('  major iteration', i_major)
116    print('    major values:   ', nit_major.current_values())
117
118    for j_minor, nit_minor in enumerate(nidx_minor):
119        nit = nit_major + nit_minor
120        print('      full values %i:'%j_minor, nit.current_values())
121
122    print()
123
124print('Test 4d index and separated iteration: try to mix dependent indices')
125nidx_major+=nidx_e
126for i_major, nit_major in enumerate(nidx_major):
127    print('  major iteration', i_major)
128    print('    major values:   ', nit_major.current_values())
129
130    for j_minor, nit_minor in enumerate(nidx_minor):
131        nit = nit_major + nit_minor
132        print('      full values %i:     '%j_minor, nit.current_values())
133        print('      formatted string %i:'%j_minor, nit.current_format('Element {element} in group {group}'))
134
135    print()