Improve structure function computation
Created originally on Bitbucket by avmo (Ashwin Vishnu)
Was already merged in Bitbucket before import, marked as merged by the import user
WORK IN PROGRESS. Painfully slow, memory intensive right now. Below is the profiling of a function similar to sim.output.increments.plot
%%prun
_rxs_str_func(sim, [2], 0, 100, 0.5, ["ux"], cache=False)plot(tmin=0, tmax=100, delta_t=0.50)
plot structure functions
tmin = 0 ; tmax = 49.6035 ; delta_t = 0.501045
imin = 0 ; imax = 99 ; delta_i = 1
724205 function calls in 4.035 seconds
Ordered by: internal time
ncalls tottime percall cumtime percall filename:lineno(function)
1 1.205 1.205 3.979 3.979 increments.py:342(load_pdf_from_file)
96960 1.107 0.000 1.107 0.000 increments.py:120(compute_values_inc)
96000 0.703 0.000 0.703 0.000 {built-in method numpy.core.multiarray.interp}
504 0.306 0.001 0.417 0.001 dataset.py:399(__getitem__)
96000 0.183 0.000 1.145 0.000 function_base.py:1179(interp)
96000 0.101 0.000 0.133 0.000 type_check.py:250(iscomplexobj)
96003 0.082 0.000 0.082 0.000 {built-in method numpy.core.multiarray.array}
506 0.058 0.000 0.104 0.000 group.py:168(__getitem__)
1 0.053 0.053 0.053 0.053 {built-in method fluidsim.base.output.util_pythran.strfunc_from_pdf}
96000 0.045 0.000 0.127 0.000 numeric.py:433(asarray)
504 0.036 0.000 0.041 0.000 dataset.py:317(__init__)
2516 0.032 0.000 0.033 0.000 dataset.py:221(shape)
96002 0.031 0.000 0.031 0.000 {built-in method builtins.issubclass}
505 0.014 0.000 0.014 0.000 {method 'reduce' of 'numpy.ufunc' objects}
504 0.008 0.000 0.018 0.000 selections.py:429(_handle_simple)
504 0.007 0.000 0.008 0.000 selections.py:406(_expand_ellipsis)
508 0.006 0.000 0.006 0.000 base.py:80(is_empty_dataspace)
504 0.005 0.000 0.024 0.000 selections.py:250(__getitem__)
504 0.005 0.000 0.005 0.000 selections.py:147(__init__)
508 0.005 0.000 0.005 0.000 dataset.py:37(readtime_dtype)
508 0.004 0.000 0.004 0.000 {built-in method h5py.h5t.py_create}
504 0.004 0.000 0.037 0.000 selections.py:27(select)
504 0.003 0.000 0.003 0.000 filters.py:209(get_filters)
504 0.003 0.000 0.019 0.000 fromnumeric.py:64(_wrapreduction)
8082 0.003 0.000 0.003 0.000 {built-in method builtins.isinstance}
504 0.002 0.000 0.008 0.000 selections.py:244(__init__)
1012 0.001 0.000 0.001 0.000 {built-in method builtins.getattr}
7053 0.001 0.000 0.001 0.000 {built-in method builtins.len}
5555 0.001 0.000 0.001 0.000 base.py:236(id)
510 0.001 0.000 0.002 0.000 base.py:109(_e)
504 0.001 0.000 0.020 0.000 fromnumeric.py:2478(prod)
530 0.001 0.000 0.002 0.000 <frozen importlib._bootstrap>:416(parent)
504 0.001 0.000 0.001 0.000 selections.py:168(nselect)
512 0.001 0.000 0.001 0.000 {method 'encode' of 'str' objects}
504 0.001 0.000 0.001 0.000 selections.py:475(_translate_slice)
504 0.001 0.000 0.021 0.000 fromnumeric.py:3163(product)
530 0.001 0.000 0.001 0.000 {method 'rpartition' of 'str' objects}
4 0.001 0.000 0.001 0.000 {built-in method numpy.core.multiarray.zeros}
100 0.001 0.000 0.001 0.000 {method 'reshape' of 'numpy.ndarray' objects}
508 0.001 0.000 0.001 0.000 base.py:269(__init__)
1 0.001 0.001 0.001 0.001 {built-in method numpy.core.multiarray.arange}
504 0.001 0.000 0.001 0.000 {built-in method builtins.sum}
504 0.001 0.000 0.001 0.000 dataset.py:418(<genexpr>)
4516 0.001 0.000 0.001 0.000 {method 'append' of 'list' objects}
1008 0.000 0.000 0.001 0.000 dataset.py:419(<genexpr>)
2520 0.000 0.000 0.000 0.000 selections.py:163(shape)
1008 0.000 0.000 0.000 0.000 selections.py:267(<genexpr>)
504 0.000 0.000 0.000 0.000 {method 'indices' of 'slice' objects}
500 0.000 0.000 0.000 0.000 selections.py:462(_translate_int)
4 0.000 0.000 0.000 0.000 attrs.py:56(__getitem__)
508 0.000 0.000 0.000 0.000 selections.py:409(<genexpr>)
1 0.000 0.000 4.035 4.035 base.py:245(_rxs_str_func)
504 0.000 0.000 0.000 0.000 {method 'items' of 'dict' objects}
1008 0.000 0.000 0.000 0.000 selections.py:239(mshape)
504 0.000 0.000 0.000 0.000 selections.py:158(id)
1 0.000 0.000 0.000 0.000 files.py:321(close)
506 0.000 0.000 0.000 0.000 base.py:97(_lapl)
504 0.000 0.000 0.000 0.000 {method 'extend' of 'list' objects}
2 0.000 0.000 0.000 0.000 files.py:89(make_fapl)
2 0.000 0.000 0.000 0.000 files.py:122(make_fid)
1 0.000 0.000 4.035 4.035 <string>:2(<module>)
1 0.000 0.000 4.035 4.035 {built-in method builtins.exec}
5 0.000 0.000 0.000 0.000 socket.py:334(send)
1 0.000 0.000 0.000 0.000 files.py:334(<listcomp>)
3 0.000 0.000 0.000 0.000 {method 'format' of 'str' objects}
2 0.000 0.000 0.000 0.000 files.py:273(__init__)
5 0.000 0.000 0.000 0.000 iostream.py:195(schedule)
4 0.000 0.000 0.000 0.000 base.py:262(attrs)
1 0.000 0.000 0.000 0.000 files.py:333(<listcomp>)
1 0.000 0.000 0.000 0.000 files.py:357(__exit__)
4 0.000 0.000 0.000 0.000 {method 'argmin' of 'numpy.ndarray' objects}
4 0.000 0.000 0.000 0.000 iostream.py:382(write)
4 0.000 0.000 0.000 0.000 group.py:34(__init__)
1 0.000 0.000 0.000 0.000 _methods.py:58(_mean)
1 0.000 0.000 0.053 0.053 increments.py:324(strfunc_from_pdf)
4 0.000 0.000 0.000 0.000 <frozen importlib._bootstrap>:997(_handle_fromlist)
4 0.000 0.000 0.000 0.000 attrs.py:51(__init__)
4 0.000 0.000 0.000 0.000 fromnumeric.py:49(_wrapfunc)
2 0.000 0.000 0.000 0.000 {built-in method builtins.print}
1 0.000 0.000 0.000 0.000 {built-in method builtins.round}
2 0.000 0.000 0.000 0.000 compat.py:103(filename_encode)
1 0.000 0.000 0.000 0.000 _methods.py:48(_count_reduce_items)
4 0.000 0.000 0.000 0.000 {built-in method builtins.abs}
4 0.000 0.000 0.000 0.000 fromnumeric.py:1040(argmin)
5 0.000 0.000 0.000 0.000 {method 'acquire' of '_thread.lock' objects}
1 0.000 0.000 0.000 0.000 fromnumeric.py:2817(mean)
5 0.000 0.000 0.000 0.000 threading.py:1104(is_alive)
2 0.000 0.000 0.000 0.000 os.py:794(fsencode)
5 0.000 0.000 0.000 0.000 iostream.py:93(_event_pipe)
4 0.000 0.000 0.000 0.000 iostream.py:307(_is_master_process)
5 0.000 0.000 0.000 0.000 threading.py:1062(_wait_for_tstate_lock)
4 0.000 0.000 0.000 0.000 {built-in method posix.getpid}
4 0.000 0.000 0.000 0.000 iostream.py:320(_schedule_flush)
9 0.000 0.000 0.000 0.000 {built-in method builtins.hasattr}
5 0.000 0.000 0.000 0.000 threading.py:506(is_set)
1 0.000 0.000 0.000 0.000 {method 'disable' of '_lsprof.Profiler' objects}
4 0.000 0.000 0.000 0.000 {built-in method posix.fspath}
1 0.000 0.000 0.000 0.000 files.py:353(__enter__)
1 0.000 0.000 0.000 0.000 numeric.py:504(asanyarray)
1 0.000 0.000 0.000 0.000 {method 'keys' of 'dict' objects}
5 0.000 0.000 0.000 0.000 {method 'append' of 'collections.deque' objects}