performance - Measure time of a function with arguments in Python

Question

I am trying to measure the time of raw_queries(...), unsuccessfully so far. I found that I should use the timeit module. The problem is that I can't (= I don't know how) pass the arguments to the function from the environment.

Important note: Before calling raw_queries, we have to execute phase2() (environment initialization).

Side note: The code is in Python 3.

def raw_queries(queries, nlp):
    """ Submit queries without getting visual response """

    for q in queries:
        nlp.query(q)

def evaluate_queries(queries, nlp):
    """ Measure the time that the queries need to return their results """

    t = Timer("raw_queries(queries, nlp)", "?????")
    print(t.timeit())

def phase2():
    """ Load dictionary to memory and subsequently submit queries """

    # prepare Linguistic Processor to submit it the queries
    all_files = get_files()
    b = LinguisticProcessor(all_files)
    b.loadDictionary()

    # load the queries
    queries_file = 'queries.txt'
    queries = load_queries(queries_file)

if __name__ == '__main__':
    phase2()

Thanks for any help.

UPDATE: We can call phase2() using the second argument of Timer. The problem is that we need the arguments (queries, nlp) from the environment.

UPDATE: The best solution so far, with unutbu's help (only what has changed):

def evaluate_queries():
    """ Measure the time that the queries need to return their results """

    t = Timer("main.raw_queries(queries, nlp)", "import main;
        (queries,nlp)=main.phase2()")

    sf = 'Execution time: {} ms'
    print(sf.format(t.timeit(number=1000)))


def phase2():
    ...

    return queries, b


def main():
    evaluate_queries()

if __name__ == '__main__':
    main()

Answer 1

First, never use the time module to time functions. It can easily lead to wrong conclusions. See timeit versus timing decorator for an example.

The easiest way to time a function call is to use IPython's %timeit command. There, you simply start an interactive IPython session, call phase2(), define queries, and then run

%timeit raw_queries(queries,nlp)

The second easiest way that I know to use timeit is to call it from the command-line:

python -mtimeit -s"import test; queries=test.phase2()" "test.raw_queries(queries)"

(In the command above, I assume the script is called test.py)

The idiom here is

python -mtimeit -s"SETUP_COMMANDS" "COMMAND_TO_BE_TIMED"

To be able to pass queries to the raw_queries function call, you have to define the queries variable. In the code you posted queries is defined in phase2(), but only locally. So to setup queries as a global variable, you need to do something like have phase2 return queries:

def phase2():
    ...
    return queries

If you don't want to mess up phase2 this way, create a dummy function:

def phase3():
    # Do stuff like phase2() but return queries
    return queries