spike.util package¶
Submodules¶
spike.util.ConfigForm module¶
spike.util.ValErr module¶
untitled.py
Created by Marc-André on 2011-03-20. Copyright (c) 2011 IGBMC. All rights reserved.
-
spike.util.ValErr.
ValErr
(v, e=0.0)[source]¶ nice print of a value with error bars
values are rounded to significative digits >>> print ValErr(1234.567,123.4) 1230 +/- 123
can be called with a pair of values or a tuple : >>> V=(1234.567,123.4); print ValErr(V) 1230 +/- 123
values are rounded to significative digits >>> print ValErr(7654.321,1234) 7700 +/- 1230 >>> print ValErr(7654.321,123.4) 7650 +/- 123 >>> print ValErr(7654.321,12.34) 7654 +/- 12
format depends on error >>> print ValErr(1234.567,12.34) 1235 +/- 12 >>> print ValErr(1234.567,1.234) 1234.6 +/- 1.2 >>> print ValErr(1234.567,0.1234) 1234.57 +/- 0.12 >>> print ValErr(1234.567,0.01234) 1234.567 +/- 0.012
spike.util.compat module¶
Compatibility methods and classes for the progressbar module.
spike.util.counter module¶
counter.py
This utility declare two useful decorators
counting and timeit
usage : @counting def func()
…
- counting
The number of time the decorated function is called is counted, and the count is printed at the end of the prgm execution
- timeit :
As counting, but additionally, the total (elapsed) time spent in the function is totalized and printed at the end of the prgm
Created by Marc-André on 2012-11-05.
Copyright (c) 2012 IGBMC. All rights reserved.
-
class
spike.util.counter.
Counter
(ini=0)[source]¶ Bases:
object
defines an incremental counter c = Counter() # actually defines a function c() v = c() # each time c() is called, the internal value is incremented value is found at c.count
-
spike.util.counter.
counting
(func_to_decorate)[source]¶ define a decorator that count the nmber of function/method call, and display at the end of the prgm
spike.util.debug_tools module¶
Created by Lionel Chiron 07/10/2013 Copyright (c) 2013 __NMRTEC__. All rights reserved. decorate the class with decclassdebugging and in each method to be debugged make : if debug(self) : instructions..
-
class
spike.util.debug_tools.
Test_debug_tools
(methodName='runTest')[source]¶ Bases:
unittest.case.TestCase
-
addClassCleanup_debug
= False¶
-
addCleanup_debug
= False¶
-
addTypeEqualityFunc_debug
= False¶
-
assertAlmostEqual_debug
= False¶
-
assertAlmostEquals_debug
= False¶
-
assertCountEqual_debug
= False¶
-
assertDictContainsSubset_debug
= False¶
-
assertDictEqual_debug
= False¶
-
assertEqual_debug
= False¶
-
assertEquals_debug
= False¶
-
assertFalse_debug
= False¶
-
assertGreaterEqual_debug
= False¶
-
assertGreater_debug
= False¶
-
assertIn_debug
= False¶
-
assertIsInstance_debug
= False¶
-
assertIsNone_debug
= False¶
-
assertIsNotNone_debug
= False¶
-
assertIsNot_debug
= False¶
-
assertIs_debug
= False¶
-
assertLessEqual_debug
= False¶
-
assertLess_debug
= False¶
-
assertListEqual_debug
= False¶
-
assertLogs_debug
= False¶
-
assertMultiLineEqual_debug
= False¶
-
assertNotAlmostEqual_debug
= False¶
-
assertNotAlmostEquals_debug
= False¶
-
assertNotEqual_debug
= False¶
-
assertNotEquals_debug
= False¶
-
assertNotIn_debug
= False¶
-
assertNotIsInstance_debug
= False¶
-
assertNotRegex_debug
= False¶
-
assertNotRegexpMatches_debug
= False¶
-
assertRaisesRegex_debug
= False¶
-
assertRaisesRegexp_debug
= False¶
-
assertRaises_debug
= False¶
-
assertRegex_debug
= False¶
-
assertRegexpMatches_debug
= False¶
-
assertSequenceEqual_debug
= False¶
-
assertSetEqual_debug
= False¶
-
assertTrue_debug
= False¶
-
assertTupleEqual_debug
= False¶
-
assertWarnsRegex_debug
= False¶
-
assertWarns_debug
= False¶
-
assert__debug
= False¶
-
countTestCases_debug
= False¶
-
debug_debug
= False¶
-
defaultTestResult_debug
= False¶
-
doClassCleanups_debug
= False¶
-
doCleanups_debug
= False¶
-
failIfAlmostEqual_debug
= False¶
-
failIfEqual_debug
= False¶
-
failIf_debug
= False¶
-
failUnlessAlmostEqual_debug
= False¶
-
failUnlessEqual_debug
= False¶
-
failUnlessRaises_debug
= False¶
-
failUnless_debug
= False¶
-
fail_debug
= False¶
-
failureException_debug
= False¶
-
hip_debug
= False¶
-
hop_debug
= False¶
-
id_debug
= False¶
-
longMessage_debug
= False¶
-
maxDiff_debug
= False¶
-
pr
(var, message='')¶ print
-
run_debug
= False¶
-
setUpClass_debug
= False¶
-
setUp_debug
= False¶
-
shortDescription_debug
= False¶
-
skipTest_debug
= False¶
-
subTest_debug
= False¶
-
tearDownClass_debug
= False¶
-
tearDown_debug
= False¶
-
test_debug_debug
= False¶
-
-
spike.util.debug_tools.
dec_class_pr
(cl)[source]¶ Decorates the class for printing the local variables
spike.util.dynsubplot module¶
Code permitting to make easily subplot without having to anticipate the number of plot. Just the number of columns is needed. typical syntax is :
sub = subpl(nbsub_h = 2) # suplot organized in two columns sub.next() # adding new subplot sub.plot(np.arange(6), np.arange(6)**2, ‘g’, label = ‘one’) # first plot in first subplot sub.title(‘One’)
-
class
spike.util.dynsubplot.
Test_dynsubplot
(methodName='runTest')[source]¶ Bases:
unittest.case.TestCase
Unittests for dynsubplot with four subplots. First plot contains 2 plots.
-
spike.util.dynsubplot.
add_class
(l)[source]¶ Each method in the list l is associated to a method defined with add_meth. This method as the same name as the pyplot method, but it has a different function. It will append the arguments passed in the plotting code to a list “lmeth” which will contains the arguments. Those
-
spike.util.dynsubplot.
add_meth
(attr)[source]¶ Evaluates methods created in dec_class. Replaces pyplot method by methods which keep the arguments. for example, subpl.title() will stock its arguments in the list “subpl.ltitle”
-
class
spike.util.dynsubplot.
subpl
(nbsub_h=1, which_plot=None)[source]¶ Bases:
object
Iterator for returning automatically the number of the subplot iteratively. nbsub_h is the number of subplot horizontally which_plot is the kind of plot, fake plot or matplotlib pyplot plot..
-
classmethod
ax
(*args, **kwargs)¶ Appends the useful arguments to the list “subpl.lmeth” instead of executing pyplot method “meth”
-
l_num
= []¶
-
lax
= []¶
-
lplot
= []¶
-
ltitle
= []¶
-
lxlabel
= []¶
-
lxticklabels
= []¶
-
lylabel
= []¶
-
make_str_kargs
(func, i)[source]¶ Extracts the list containing the arguments from sublist [1] of each method, in the list “self.lmeth”.
-
next
()[source]¶ Increments the number of plots “subpl.numplot” And calculates the number of vertical lines.
-
numplot
= 0¶
-
classmethod
plot
(*args, **kwargs)¶ Appends the useful arguments to the list “subpl.lmeth” instead of executing pyplot method “meth”
-
show
()[source]¶ Takes all the list of arguments from the fake methods sub.meth And evaluates them one after the other with the correct pyplot corresponding method.
-
classmethod
title
(*args, **kwargs)¶ Appends the useful arguments to the list “subpl.lmeth” instead of executing pyplot method “meth”
-
classmethod
xlabel
(*args, **kwargs)¶ Appends the useful arguments to the list “subpl.lmeth” instead of executing pyplot method “meth”
-
classmethod
xticklabels
(*args, **kwargs)¶ Appends the useful arguments to the list “subpl.lmeth” instead of executing pyplot method “meth”
-
classmethod
ylabel
(*args, **kwargs)¶ Appends the useful arguments to the list “subpl.lmeth” instead of executing pyplot method “meth”
-
classmethod
spike.util.hashcheck module¶
Program for controlling if files are not deteriorated during transmission. Usage : write in command line “python hashcheck.py directory-pathname” Lionel Chiron, 11 oct 2012 copyright NMRTEC S.A.S.
-
class
spike.util.hashcheck.
Test_hashcheck
(methodName='runTest')[source]¶ Bases:
unittest.case.TestCase
-
spike.util.hashcheck.
hash_make_or_control
(rootpath='.', verbose=False)[source]¶ Makes the file hashfile in the directory hash if it doesn’t exists. If the hash directory exists, check if files are ok or not.
spike.util.log_all module¶
Created by Lionel Chiron 18/10/2013 Copyright (c) 2013 __NMRTEC__. All rights reserved.
Utility for reporting error from standard error output via gmail. When sys.stderr performs a “write”, a mail is sent with a report. Typical syntax is:
import sys sys.stderr = Logger() f = open(“fff.jpg”)
If the picture “fff.jpg” doesn’t exist, an arror message is sent by mail.
spike.util.log_stderr module¶
Created by Lionel Chiron 18/10/2013 Copyright (c) 2013 __NMRTEC__. All rights reserved.
Utility for reporting error from standard error output via gmail. When sys.stderr performs a “write”, a mail is sent with a report. Typical syntax is:
import sys sys.stderr = Logger() f = open(“fff.jpg”)
If the picture “fff.jpg” doesn’t exist, an arror message is sent by mail.
spike.util.log_stdout module¶
spike.util.mail module¶
spike.util.mail_error_logger module¶
Created by Lionel Chiron 18/10/2013 Copyright (c) 2013 __NMRTEC__. All rights reserved.
Utility for reporting error from standard error output via gmail. When sys.stderr performs a “write”, a mail is sent with a report. Typical syntax is:
import sys sys.stderr = Logger() f = open(“fff.jpg”)
If the picture “fff.jpg” doesn’t exist, an arror message is sent by mail.
spike.util.mpiutil module¶
mpiutil.py
Utilities for using MPI
defines MPI_size : number of process in the pool MPI_rank : rank of the current process
typical use is presented in test_server_worker()
- in main do :
- if MPI_rank == 0:
compute() # your code, uses mpiutil.enum_imap(myfunction, xarg)
- else:
mpiutil.slave() # slave is a generic function used to compute myfunction( arg )
create a function to do the processing def myfunction(arg)
return whatever
- and in compute() :
# create an iterator over arguments M = 1000 # size of the pb result = M*[0.0] # create room for storing results (if needed) xarg = xrange(M) # create arguments res = mpiutil.enum_imap(myfunction, xarg) # enum_imap applies myfunction to the iterator xarg for i,r in res: # collect results
result[i] = r # and store them (order is not garantied)
Warning, (i,r) will not arrive in order
Will import even in MPI is not installed, however slave() will not work The situation is reported as MPI_size = 1
Created by Marc-Andre’ on 2012-04-19. extended 2014-03-25
Copyright (c) 2012-2014 IGBMC. All rights reserved.
-
spike.util.mpiutil.
enum_imap
(function, iterator)[source]¶ applies function() to each element of iterator creates an iterator imap(function, iterator) that returns (i, function(iterator_i)) elements will be returned in an synchronous manner with no garanty on the order.
similar to enumerate( pool.imap_async(function, iterator) ) except for the order
-
spike.util.mpiutil.
funct_array_simple1
(a)[source]¶ used for testing take an array and compute 2*sqr(a) +1 plus a random nap to simulate a random processing time
-
spike.util.mpiutil.
funct_array_simple2
(a)[source]¶ used for testing take an array and compute 3*sqr(a+1) plus a random nap to simulate a random processing time
-
spike.util.mpiutil.
initialize
(func=<function _nill>)[source]¶ bring all processes to a starting barrier to be called by master process (rank == 0)
spike.util.near_prime module¶
spike.util.progrbarUi module¶
spike.util.progressbar module¶
Main ProgressBar class.
-
class
spike.util.progressbar.
ProgressBar
(maxval=None, widgets=None, term_width=None, poll=1, left_justify=True, fd=<_io.TextIOWrapper name='<stderr>' mode='w' encoding='utf-8'>)[source]¶ Bases:
object
The ProgressBar class which updates and prints the bar.
A common way of using it is like: >>> pbar = ProgressBar().start() >>> for i in range(100): … # do something … pbar.update(i+1) … >>> pbar.finish()
You can also use a ProgressBar as an iterator: >>> progress = ProgressBar() >>> for i in progress(some_iterable): … # do something …
Since the progress bar is incredibly customizable you can specify different widgets of any type in any order. You can even write your own widgets! However, since there are already a good number of widgets you should probably play around with them before moving on to create your own widgets.
The term_width parameter represents the current terminal width. If the parameter is set to an integer then the progress bar will use that, otherwise it will attempt to determine the terminal width falling back to 80 columns if the width cannot be determined.
When implementing a widget’s update method you are passed a reference to the current progress bar. As a result, you have access to the ProgressBar’s methods and attributes. Although there is nothing preventing you from changing the ProgressBar you should treat it as read only.
- Useful methods and attributes include (Public API):
currval: current progress (0 <= currval <= maxval)
maxval: maximum (and final) value
finished: True if the bar has finished (reached 100%)
start_time: the time when start() method of ProgressBar was called
- seconds_elapsed: seconds elapsed since start_time and last call to
update
percentage(): progress in percent [0..100]
-
currval
¶
-
fd
¶
-
finished
¶
-
last_update_time
¶
-
left_justify
¶
-
maxval
¶
-
next
()¶
-
next_update
¶
-
num_intervals
¶
-
property
percent
¶ Returns the progress as a percentage.
-
poll
¶
-
seconds_elapsed
¶
-
signal_set
¶
-
start
()[source]¶ Starts measuring time, and prints the bar at 0%.
It returns self so you can use it like this: >>> pbar = ProgressBar().start() >>> for i in range(100): … # do something … pbar.update(i+1) … >>> pbar.finish()
-
start_time
¶
-
term_width
¶
-
update_interval
¶
-
widgets
¶
spike.util.readwrite_tools module¶
-
spike.util.readwrite_tools.
read
(f)[source]¶ reads Orbitrap .dat FID files returns a numpy array with the FID
spike.util.sendgmail module¶
Sending informations about result etc with Gmail recipient (to) and attached document (attach) can be a list
spike.util.shift_comments module¶
Usage: python shift_comment.py file.py nbcolumn nb column is the column nb at which we want the comment to be placed. If no nb is given will shift the comments from the end of the line at a regular intervall.
spike.util.signal_tools module¶
Created by Lionel Chiron 18/10/2013 Copyright (c) 2013 __NMRTEC__. All rights reserved. Various tools for performing signal processing (SNR calculation,
synthetic Fids with noise, noise level etc… )
-
class
spike.util.signal_tools.
SIGNAL_NOISE
(lenfid, nbpeaks, amplitude, noise, LB=1.11, shape='triangular', noisetype='additive', shift=0, realfid=False, trunc=None, seed=None, baseline=None)[source]¶ Bases:
object
Tool for generating SIGNAL with/without NOISE.
- input:
lenfid : length of the Fid nbpeaks : nb of frequencies amplitude : reference amplitude for signal LB : exponential apodisation factor noise : noise amplitude shape : shape of the spectrum. Available : triangular, list. noisetype : kind of noise, ie : additive, multiplicative etc… shift : shift of the frequencies, the shift is expressed in points. trunc : truncaton ratio for the Fid. seed : for using a precise seed for random processes. baseline : boolean for specifying if a baseline is added to the spectrum
- output:
Noisy signal : self.fid Signal without noise : self.fid0 Noisy spectrum : self.spec Spectrum without noise : self.spec0
- eg1: # simplest usage
sig = SIGNAL_NOISE( lenfid= 1000, nbpeaks = 20, amplitude = 27, noise = 5)
- eg2: # with a list of amplitudes
lamplit = list(np.random.randn(20)*10) sig = st.SIGNAL_NOISE( lenfid= 10000, nbpeaks = 20, amplitude = lamplit, noise = 4, shape = “list”)
-
property
iSNR
¶ Reference SNR
-
prepare_amplitudes
()[source]¶ Makes the list of amplitudes according to the case. triangular: makes a triangular shape beginning at amplitude “self.amplitude” with slope “self.slope” list: retrieves amplitudes from given list of amplitudes.
-
property
spec
¶ Noisy spectrum
-
property
spec0
¶ Spectrum wihtout noise
-
spike.util.signal_tools.
SNR
(dataset, nbseg=20, peak='max', unit=None)[source]¶ determines the signal to noise of NPKData, by dividing le largest peak witht the noiselevel handles complex spectra, taking the noiselevel and the peak on the real part noise is evaluated by segmenting the data-set in nbseg peices, check in findnoiselevel, peak is either max or an index on which to evaluate the signal intensity if unit is None, value is in ration, if uint is dB, value is in dB
-
class
spike.util.signal_tools.
Test_Units
(methodName='runTest')[source]¶ Bases:
unittest.case.TestCase
Unittests test_signoise test_signoise_truncated test_noiselevel test_multitests_urQRdtrick test_multitests_urQRd_npk
-
spike.util.signal_tools.
fid_signoise
(nbpeaks, ampl, lengthfid, noise, shift=0, shape='triangular', seed=True)[source]¶ Obsolete
Build fid with triangular spectrum from number of peaks : nbpeaks, amplitude : ampl, the length of the fid : lengthfid, and the noise level : noise. if shape is “triangular”, uses ampl as minimum amplitude. if shape is “list”, uses the given list to make the amplitudes. The seed of random generator can be activated or deactivated with boolean ‘seed’
-
spike.util.signal_tools.
filtering
(sig, window=None)[source]¶ Filter for real signal window : list [f1, f2], eg: [0.1, 0.3], 0 < f < 1
-
spike.util.signal_tools.
findnoiselevel
(fid, nbseg=20)[source]¶ Routine for determining the noise level from a numpy buffer “fid” cut the data in segments then make the standard deviation return makes the average on the 25% lowest segments
nbseg=10 nb of segment to cut the spectrum
-
spike.util.signal_tools.
findnoiselevel2
(fid, nbseg=20)[source]¶ (findnoiselevel seems more reliable) Routine for determining the noise level from a numpy buffer “fid” cut the data in segments then make the standard deviation and returns the smallest deviation
nbseg=10 nb of segment to cut the spectrum
-
spike.util.signal_tools.
findnoiselevel_2D
(data_array, nbseg=20, nbrow=200)[source]¶ measure noise level on a 2D spectrum
-
spike.util.signal_tools.
findnoiselevel_2D_slice
(data_array, ratio=1)[source]¶ Routine for finding noise level by dividing the 2D spectrum in vertical slices.
-
spike.util.signal_tools.
findnoiselevel_offset
(fid, nbseg=10)[source]¶ Routine for determining the noise level and mean value (considering a flat signal with gaussian noise and some outliers in the signal) cut the data in segments then make the standard deviation for each segment compares the std dev with the average one and eliminates segments giving a std dev above the mean finally makes the average on the sorted segments. nbseg=10 nb of segment to cut the spectrum
spike.util.simple_logger module¶
spike.util.simple_logger2 module¶
Created by Lionel Chiron 18/10/2013 Copyright (c) 2013 __NMRTEC__. All rights reserved.
Utility for copying Standard Error (stderr) and Standard Output (stdout) to a file
-
class
spike.util.simple_logger2.
TeeLogger
(log_name='log_file.log', erase=False, date_in=True, err_prefix='___')[source]¶ Bases:
object
Simple logger.
TeeLogger(log_name = “log_file.log”, erase = False, date_in = True, err_prefix = “___”) or TeeLogger()
copies standard Error (stderr) and Standard Output (stdout) to a file
log_name : name of the log file, if no name given, it is called “log_file.log” erase : if True erase the existing log file with the same name. date_in: in True (defualt) date is indicated in log file err_prefix stderr is prefixed with this string
spike.util.widgets module¶
Default ProgressBar widgets.
-
class
spike.util.widgets.
AdaptiveETA
(format='Elapsed Time: %s')[source]¶ Bases:
spike.util.widgets.Timer
Widget which attempts to estimate the time of arrival.
- Uses a weighted average of two estimates:
ETA based on the total progress and time elapsed so far
ETA based on the progress as per the last 10 update reports
The weight depends on the current progress so that to begin with the total progress is used and at the end only the most recent progress is used.
-
NUM_SAMPLES
= 10¶
-
TIME_SENSITIVE
= True¶
-
format_string
¶
-
class
spike.util.widgets.
AnimatedMarker
(markers='|/-\\')[source]¶ Bases:
spike.util.widgets.Widget
An animated marker for the progress bar which defaults to appear as if it were rotating.
-
curmark
¶
-
markers
¶
-
-
class
spike.util.widgets.
Bar
(marker='#', left='|', right='|', fill=' ', fill_left=True)[source]¶ Bases:
spike.util.widgets.WidgetHFill
A progress bar which stretches to fill the line.
-
fill
¶
-
fill_left
¶
-
left
¶
-
marker
¶
-
right
¶
-
-
class
spike.util.widgets.
BouncingBar
(marker='#', left='|', right='|', fill=' ', fill_left=True)[source]¶ Bases:
spike.util.widgets.Bar
-
fill
¶
-
fill_left
¶
-
left
¶
-
marker
¶
-
right
¶
-
-
class
spike.util.widgets.
Counter
(format='%d')[source]¶ Bases:
spike.util.widgets.Widget
Displays the current count.
-
format_string
¶
-
-
class
spike.util.widgets.
ETA
(format='Elapsed Time: %s')[source]¶ Bases:
spike.util.widgets.Timer
Widget which attempts to estimate the time of arrival.
-
TIME_SENSITIVE
= True¶
-
format_string
¶
-
-
class
spike.util.widgets.
FileTransferSpeed
(unit='B')[source]¶ Bases:
spike.util.widgets.Widget
Widget for showing the transfer speed (useful for file transfers).
-
FORMAT
= '%6.2f %s%s/s'¶
-
PREFIXES
= ' kMGTPEZY'¶
-
unit
¶
-
-
class
spike.util.widgets.
FormatLabel
(format)[source]¶ Bases:
spike.util.widgets.Timer
Displays a formatted label.
-
format_string
¶
-
mapping
= {'elapsed': ('seconds_elapsed', <function Timer.format_time>), 'finished': ('finished', None), 'last_update': ('last_update_time', None), 'max': ('maxval', None), 'seconds': ('seconds_elapsed', None), 'start': ('start_time', None), 'value': ('currval', None)}¶
-
-
class
spike.util.widgets.
Percentage
[source]¶ Bases:
spike.util.widgets.Widget
Displays the current percentage as a number with a percent sign.
-
class
spike.util.widgets.
ReverseBar
(marker='#', left='|', right='|', fill=' ', fill_left=False)[source]¶ Bases:
spike.util.widgets.Bar
A bar which has a marker which bounces from side to side.
-
fill
¶
-
fill_left
¶
-
left
¶
-
marker
¶
-
right
¶
-
-
spike.util.widgets.
RotatingMarker
¶ alias of
spike.util.widgets.AnimatedMarker
-
class
spike.util.widgets.
SimpleProgress
(sep=' of ')[source]¶ Bases:
spike.util.widgets.Widget
Returns progress as a count of the total (e.g.: “5 of 47”).
-
sep
¶
-
-
class
spike.util.widgets.
Timer
(format='Elapsed Time: %s')[source]¶ Bases:
spike.util.widgets.Widget
Widget which displays the elapsed seconds.
-
TIME_SENSITIVE
= True¶
-
format_string
¶
-
-
class
spike.util.widgets.
Widget
[source]¶ Bases:
abc.AbstractWidget
The base class for all widgets.
The ProgressBar will call the widget’s update value when the widget should be updated. The widget’s size may change between calls, but the widget may display incorrectly if the size changes drastically and repeatedly.
The boolean TIME_SENSITIVE informs the ProgressBar that it should be updated more often because it is time sensitive.
-
TIME_SENSITIVE
= False¶
-
-
class
spike.util.widgets.
WidgetHFill
[source]¶ Bases:
spike.util.widgets.Widget
The base class for all variable width widgets.
This widget is much like the hfill command in TeX, it will expand to fill the line. You can use more than one in the same line, and they will all have the same width, and together will fill the line.