some profiling utils I want to keep reusing

profile_call[source]

profile_call(decorator_arg, fn_name=None)

decorator to profile a method with an optional name - stores data in PROF DATA

print_prof_data(fname=None)

print out profile data

clear_prof_data[source]

clear_prof_data(fname=None)

clear out profile data

get_prof_data[source]

get_prof_data(name)

get profile data for name

start_record[source]

start_record(name)

start recording time for name

end_record[source]

end_record(name)

end recording time and add elapsed time to profile data

is_recording[source]

is_recording(name)

return True if recording for function name started but not yet completed

save_prof_data[source]

save_prof_data(file_name, overwrite_file=True)

save profile data to file_name, overwrite_file=True overwrites existing file

load_prof_data[source]

load_prof_data(file_name, overwrite_prof_data=True)

load profile data from file_name, overwrite_prof_data overwrites existing profile data

Usage Examples

@profile_call
def test_func(t=2.0):
    time.sleep(1.0)

for i in range(10):
    test_func(i)

print_prof_data()
Function test_func called 10 times.
Execution time max: 1.005, average: 1.002
times = get_prof_data('test_func'); times
[1.0005080699920654,
 1.0043628215789795,
 1.0049660205841064,
 1.0018970966339111,
 1.000082015991211,
 1.000168800354004,
 1.0024340152740479,
 1.0003981590270996,
 1.0037879943847656,
 1.004991054534912]
PROF_DATA
{'test_func': [10,
  [1.0005080699920654,
   1.0043628215789795,
   1.0049660205841064,
   1.0018970966339111,
   1.000082015991211,
   1.000168800354004,
   1.0024340152740479,
   1.0003981590270996,
   1.0037879943847656,
   1.004991054534912],
  0]}
clear_prof_data()
@profile_call('wachacha')
def test_func2(t=1.0):
    time.sleep(t)
for i in range(10):
    test_func2()
    
print_prof_data()
Function wachacha called 10 times.
Execution time max: 1.004, average: 1.002
clear_prof_data()
@profile_call
def test_func3(t=1.0):
    time.sleep(1.0)
for i in range(10):
    test_func3()
    
print_prof_data()
Function test_func3 called 10 times.
Execution time max: 1.005, average: 1.002
for i in range(10):
    start_record('timed_sleep')
    time.sleep(i)
    end_record('timed_sleep')
print_prof_data('timed_sleep')
times2 = get_prof_data('timed_sleep')
Function timed_sleep called 10 times.
Execution time max: 9.003, average: 4.502

As an alternative, the decorator can be invoked this way

sleep = profile_call(time.sleep)
for i in range(5):
    sleep(i)

print_prof_data()
Function test_func3 called 10 times.
Execution time max: 1.005, average: 1.002
Function timed_sleep called 10 times.
Execution time max: 9.003, average: 4.502
Function sleep called 5 times.
Execution time max: 4.000, average: 2.002

We can also specify just the function name to clear from the profile data

clear_prof_data('timed_sleep')
print_prof_data()
Function test_func3 called 10 times.
Execution time max: 1.005, average: 1.002
Function sleep called 5 times.
Execution time max: 4.000, average: 2.002
sleep = profile_call(time.sleep,'malumam')
for i in range(5):
    sleep(1)

print_prof_data()
Function test_func3 called 10 times.
Execution time max: 1.005, average: 1.002
Function sleep called 5 times.
Execution time max: 4.000, average: 2.002
Function malumam called 5 times.
Execution time max: 1.005, average: 1.002
import pathlib
save_file = 'my_profile_data.pickle'
if pathlib.Path(save_file).is_file(): pathlib.Path(save_file).unlink()
save_prof_data(save_file)

Assert that my_profile_data.pickle file was created.

assert pathlib.Path(save_file).is_file()
clear_prof_data()
print_prof_data()
PROF_DATA
{}
load_prof_data(save_file)
print_prof_data()
Function test_func3 called 10 times.
Execution time max: 1.005, average: 1.002
Function sleep called 5 times.
Execution time max: 4.000, average: 2.002
Function malumam called 5 times.
Execution time max: 1.005, average: 1.002
pathlib.Path(save_file).unlink()
import matplotlib.pyplot as plt
plt.plot(times);
plt.plot(times2);