AttributeError: 'NoneType' object has no attribute '_get_values_from_container' (parameter fitting attempt)

Expected behavior.

Hello everyone,

I have tried to implement parameter fitting into my model. I get this error message:

(AttributeError: 'NoneType' object has no attribute '_get_values_from_container'). 

Expected behavior:
After entering the optimization, CADET should optimize and start printing out the results of each generation.

Actual behavior

The optimization seems to start. For a very long time, nothing happens. Then, before anything is printed out, the given error message appears, and the execution fails.

Here is the complete, very long error message:


_RemoteTraceback Traceback (most recent call last)
_RemoteTraceback:
“”"
Traceback (most recent call last):
File “e:\Programme\Miniforge3\envs\cadet_workshop\Lib\site-packages\joblib\externals\loky\process_executor.py”, line 490, in _process_worker
r = call_item()
^^^^^^^^^^^
File “e:\Programme\Miniforge3\envs\cadet_workshop\Lib\site-packages\joblib\externals\loky\process_executor.py”, line 291, in call
return self.fn(*self.args, **self.kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “e:\Programme\Miniforge3\envs\cadet_workshop\Lib\site-packages\joblib\parallel.py”, line 607, in call
return [func(*args, **kwargs) for func, args, kwargs in self.items]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “e:\Programme\Miniforge3\envs\cadet_workshop\Lib\site-packages\joblib\parallel.py”, line 607, in
return [func(*args, **kwargs) for func, args, kwargs in self.items]
^^^^^^^^^^^^^^^^^^^^^
File “e:\Programme\Miniforge3\envs\cadet_workshop\Lib\site-packages\CADETProcess\optimization\optimizationProblem.py”, line 792, in target_wrapper
results = self._evaluate_individual(
^^^^^^^^^^^^^^^^^^^^^^^^^^
File “e:\Programme\Miniforge3\envs\cadet_workshop\Lib\site-packages\CADETProcess\optimization\optimizationProblem.py”, line 838, in _evaluate_individual
value = self._evaluate(x, eval_fun, force)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “e:\Programme\Miniforge3\envs\cadet_workshop\Lib\site-packages\CADETProcess\optimization\optimizationProblem.py”, line 925, in _evaluate
result = step.evaluate(current_request)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “e:\Programme\Miniforge3\envs\cadet_workshop\Lib\site-packages\CADETProcess\optimization\optimizationProblem.py”, line 4123, in call
results = self.evaluator(request, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “e:\Programme\Miniforge3\envs\cadet_workshop\Lib\site-packages\CADETProcess\simulator\simulator.py”, line 275, in simulate
results = self.simulate_n_cycles(
^^^^^^^^^^^^^^^^^^^^^^^
File “e:\Programme\Miniforge3\envs\cadet_workshop\Lib\site-packages\CADETProcess\log.py”, line 164, in wrapper
result = function(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^
File “e:\Programme\Miniforge3\envs\cadet_workshop\Lib\site-packages\CADETProcess\log.py”, line 225, in wrapper
results = function(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^
File “e:\Programme\Miniforge3\envs\cadet_workshop\Lib\site-packages\CADETProcess\log.py”, line 200, in wrapper
raise e
File “e:\Programme\Miniforge3\envs\cadet_workshop\Lib\site-packages\CADETProcess\log.py”, line 192, in wrapper
return function(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^
File “e:\Programme\Miniforge3\envs\cadet_workshop\Lib\site-packages\CADETProcess\simulator\simulator.py”, line 334, in simulate_n_cycles
return self._run(process, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “e:\Programme\Miniforge3\envs\cadet_workshop\Lib\site-packages\CADETProcess\simulator\cadetAdapter.py”, line 154, in wrapper
results = func(self, process, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “e:\Programme\Miniforge3\envs\cadet_workshop\Lib\site-packages\CADETProcess\simulator\cadetAdapter.py”, line 290, in _run
f"Simulation of {process.name} with parameters {process.config} failed."
File “e:\Programme\Miniforge3\envs\cadet_workshop\Lib\site-packages\CADETProcess\dataStructure\aggregator.py”, line 88, in repr
return f"NumpyProxyA{self._get_values_from_aggregator().repr()[1:]}"
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “e:\Programme\Miniforge3\envs\cadet_workshop\Lib\site-packages\CADETProcess\dataStructure\aggregator.py”, line 46, in _get_values_from_aggregator
return self.aggregator._get_values_from_container(self.instance, transpose=True, check=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: ‘NoneType’ object has no attribute ‘_get_values_from_container’
“”"

The above exception was the direct cause of the following exception:

AttributeError Traceback (most recent call last)
Cell In[62], line 3
1 #Optimieren
----> 3 optimization_results = optimizer.optimize(
4 optimization_problem,
5 use_checkpoint=False
6 )

File e:\Programme\Miniforge3\envs\cadet_workshop\Lib\site-packages\CADETProcess\optimization\optimizer.py:277, in OptimizerBase.optimize(self, optimization_problem, x0, save_results, results_directory, use_checkpoint, overwrite_results_directory, exist_ok, log_level, reinit_cache, delete_cache, *args, **kwargs)
274 plt.switch_backend(“agg”)
276 start = time.time()
→ 277 self._run(self.optimization_problem, x0, *args, **kwargs)
278 time_elapsed = time.time() - start
280 self.results.time_elapsed = time_elapsed

File e:\Programme\Miniforge3\envs\cadet_workshop\Lib\site-packages\CADETProcess\optimization\pymooAdapter.py:165, in PymooInterface._run(self, optimization_problem, x0)
162 X = pop.get(“X”).tolist()
164 # Evaluate objectives and report results
→ 165 algorithm.evaluator.eval(problem, pop)
167 F = pop.get(“F”).tolist()
168 if optimization_problem.n_nonlinear_constraints > 0:

File e:\Programme\Miniforge3\envs\cadet_workshop\Lib\site-packages\pymoo\core\evaluator.py:69, in Evaluator.eval(self, problem, pop, skip_already_evaluated, evaluate_values_of, count_evals, **kwargs)
65 # evaluate the solutions (if there are any)
66 if len(I) > 0:
67
68 # do the actual evaluation - call the sub-function to set the corresponding values to the population
—> 69 self._eval(problem, pop[I], evaluate_values_of, **kwargs)
71 # update the function evaluation counter
72 if count_evals:

File e:\Programme\Miniforge3\envs\cadet_workshop\Lib\site-packages\pymoo\core\evaluator.py:90, in Evaluator._eval(self, problem, pop, evaluate_values_of, **kwargs)
87 X = pop.get(“X”)
89 # call the problem to evaluate the solutions
—> 90 out = problem.evaluate(X, return_values_of=evaluate_values_of, return_as_dictionary=True, **kwargs)
92 # for each of the attributes set it to the problem
93 for key, val in out.items():

File e:\Programme\Miniforge3\envs\cadet_workshop\Lib\site-packages\pymoo\core\problem.py:257, in Problem.evaluate(self, X, return_values_of, return_as_dictionary, *args, **kwargs)
254 only_single_value = not (isinstance(X, list) or isinstance(X, np.ndarray))
256 # this is where the actual evaluation takes place
→ 257 _out = self.do(X, return_values_of, *args, **kwargs)
259 out = {}
260 for k, v in _out.items():
261
262 # copy it to a numpy array (it might be one of jax at this point)

File e:\Programme\Miniforge3\envs\cadet_workshop\Lib\site-packages\pymoo\core\problem.py:299, in Problem.do(self, X, return_values_of, *args, **kwargs)
297 self._evaluate_elementwise(X, out, *args, **kwargs)
298 else:
→ 299 self._evaluate_vectorized(X, out, *args, **kwargs)
301 # finally format the output dictionary
302 out = self._format_dict(out, len(X), return_values_of)

File e:\Programme\Miniforge3\envs\cadet_workshop\Lib\site-packages\pymoo\core\problem.py:307, in Problem._evaluate_vectorized(self, X, out, *args, **kwargs)
306 def _evaluate_vectorized(self, X, out, *args, **kwargs):
→ 307 self._evaluate(X, out, *args, **kwargs)

File e:\Programme\Miniforge3\envs\cadet_workshop\Lib\site-packages\CADETProcess\optimization\pymooAdapter.py:293, in PymooProblem._evaluate(self, X, out, *args, **kwargs)
291 opt = self.optimization_problem
292 if opt.n_objectives > 0:
→ 293 F = opt.evaluate_objectives(
294 X,
295 untransform=True,
296 get_dependent_values=True,
297 ensure_minimization=True,
298 parallelization_backend=self.parallelization_backend,
299 )
300 out[“F”] = np.array(F)
302 if opt.n_nonlinear_constraints > 0:

File e:\Programme\Miniforge3\envs\cadet_workshop\Lib\site-packages\CADETProcess\optimization\optimizationProblem.py:206, in OptimizationProblem.ensures2d..wrapper_ensures2d(self, X, *args, **kwargs)
203 X_2d = np.array(X, ndmin=2)
205 # Call and ensure results are 2D
→ 206 Y = func(self, X_2d, *args, **kwargs)
207 Y_2d = Y.reshape((len(X_2d), -1))
209 # Reshape back to original length of X

File e:\Programme\Miniforge3\envs\cadet_workshop\Lib\site-packages\CADETProcess\optimization\optimizationProblem.py:169, in OptimizationProblem.untransforms..wrapper_untransforms(self, x, untransform, *args, **kwargs)
166 if untransform:
167 x = self.untransform(x)
→ 169 return func(self, x, *args, **kwargs)

File e:\Programme\Miniforge3\envs\cadet_workshop\Lib\site-packages\CADETProcess\optimization\optimizationProblem.py:187, in OptimizationProblem.gets_dependent_values..wrapper_gets_dependent_values(self, x, get_dependent_values, *args, **kwargs)
184 if get_dependent_values:
185 x = self.get_dependent_values(x)
→ 187 return func(self, x, *args, **kwargs)

File e:\Programme\Miniforge3\envs\cadet_workshop\Lib\site-packages\CADETProcess\optimization\optimizationProblem.py:228, in OptimizationProblem.ensures_minimization..wrap..wrapper_ensures_minimization(self, ensure_minimization, *args, **kwargs)
221 @wraps(func)
222 def wrapper_ensures_minimization(
223 self: OptimizationProblem,
(…) 226 **kwargs: Any,
227 ) → Any:
→ 228 s = func(self, *args, **kwargs)
230 if ensure_minimization:
231 s = self.transform_maximization(s, scores)

File e:\Programme\Miniforge3\envs\cadet_workshop\Lib\site-packages\CADETProcess\optimization\optimizationProblem.py:1171, in OptimizationProblem.evaluate_objectives(self, X, parallelization_backend, force)
1136 @ensures2d
1137 @untransforms
1138 @gets_dependent_values
(…) 1144 force: bool = False,
1145 ) → np.ndarray:
1146 “”"
1147 Evaluate objective functions for each individual x in population X.
1148
(…) 1169 _evaluate
1170 “”"
→ 1171 return self._evaluate_population(
1172 target_functions=self.objectives,
1173 X=X,
1174 parallelization_backend=parallelization_backend,
1175 force=force,
1176 )

File e:\Programme\Miniforge3\envs\cadet_workshop\Lib\site-packages\CADETProcess\optimization\optimizationProblem.py:801, in OptimizationProblem._evaluate_population(self, target_functions, X, parallelization_backend, force)
797 self.cache.close()
799 return results
→ 801 results = parallelization_backend.evaluate(target_wrapper, X)
802 return np.array(results, ndmin=2)

File e:\Programme\Miniforge3\envs\cadet_workshop\Lib\site-packages\CADETProcess\optimization\parallelizationBackend.py:153, in Joblib.evaluate(self, function, population)
137 “”"
138 Evaluate the function in parallalel for all individuals of the population.
139
(…) 150 List of results of function evaluations.
151 “”"
152 backend = Parallel(n_jobs=self.n_cores, verbose=self.verbose)
→ 153 results = backend(delayed(function)(x) for x in population)
155 return results

File e:\Programme\Miniforge3\envs\cadet_workshop\Lib\site-packages\joblib\parallel.py:2072, in Parallel.call(self, iterable)
2066 # The first item from the output is blank, but it makes the interpreter
2067 # progress until it enters the Try/Except block of the generator and
2068 # reaches the first yield statement. This starts the asynchronous
2069 # dispatch of the tasks to the workers.
2070 next(output)
→ 2072 return output if self.return_generator else list(output)

File e:\Programme\Miniforge3\envs\cadet_workshop\Lib\site-packages\joblib\parallel.py:1682, in Parallel._get_outputs(self, iterator, pre_dispatch)
1679 yield
1681 with self._backend.retrieval_context():
→ 1682 yield from self._retrieve()
1684 except GeneratorExit:
1685 # The generator has been garbage collected before being fully
1686 # consumed. This aborts the remaining tasks if possible and warn
1687 # the user if necessary.
1688 self._exception = True

File e:\Programme\Miniforge3\envs\cadet_workshop\Lib\site-packages\joblib\parallel.py:1784, in Parallel._retrieve(self)
1778 while self._wait_retrieval():
1779 # If the callback thread of a worker has signaled that its task
1780 # triggered an exception, or if the retrieval loop has raised an
1781 # exception (e.g. GeneratorExit), exit the loop and surface the
1782 # worker traceback.
1783 if self._aborting:
→ 1784 self._raise_error_fast()
1785 break
1787 nb_jobs = len(self._jobs)

File e:\Programme\Miniforge3\envs\cadet_workshop\Lib\site-packages\joblib\parallel.py:1859, in Parallel._raise_error_fast(self)
1855 # If this error job exists, immediately raise the error by
1856 # calling get_result. This job might not exists if abort has been
1857 # called directly or if the generator is gc’ed.
1858 if error_job is not None:
→ 1859 error_job.get_result(self.timeout)

File e:\Programme\Miniforge3\envs\cadet_workshop\Lib\site-packages\joblib\parallel.py:758, in BatchCompletionCallBack.get_result(self, timeout)
752 backend = self.parallel._backend
754 if backend.supports_retrieve_callback:
755 # We assume that the result has already been retrieved by the
756 # callback thread, and is stored internally. It’s just waiting to
757 # be returned.
→ 758 return self._return_or_raise()
760 # For other backends, the main thread needs to run the retrieval step.
761 try:

File e:\Programme\Miniforge3\envs\cadet_workshop\Lib\site-packages\joblib\parallel.py:773, in BatchCompletionCallBack._return_or_raise(self)
771 try:
772 if self.status == TASK_ERROR:
→ 773 raise self._result
774 return self._result
775 finally:

AttributeError: ‘NoneType’ object has no attribute ‘_get_values_from_container’


How to produce bug (including a minimal reproducible example)

The problem occurs, when I try to fit the film diffusion for every single one of my 6 components simultaneously. I’ve tried the same model, but then excluded all comonents except the one with index 5 for film diffusion fitting. This worked.

I provide an MRE: Fit_Error_get_result_MRE.py. It requires IA-values_pH8_MRE.xlsx for the reference data. Both in the provided repository.

I’ve started to speculate that this error might be caused by at least one of the evaluation-simulations failing. Maybe you could confirm this, or else explain, what is actually happening, and eventually, how it could be fixed? That would be much appreciated.

Greetings, AN_IA

Optional link to repository (oriented around this template: GitHub - modsim/bug_report_example: Template for reporting bugs including code and data)

Hi AN_IA,

Thanks for reporting your issue.

Since the error occurs inside the joblib package, I would like to rule out any parallelisation issues. Please set optimizer.n_cores = 1 in your optimisation run and let me know if the error persists.

All the best,
Hannah

Hi,

I’m getting back to this with some delay. First of all, thanks for your reply here also.

I have performed some tests, but I had to find a time window where I could test the model with one core on my original system, where I encountered the error. I have done the following test runs of my MRE with 1 core in the fitting:

  • On a compute cluster, run time 3.5 days
  • On my original system, run time 28hrs, still running

So far, none of these yielded ANY result, be it positive or negative. Neither have I gotten any output that indicates the finishing of an optimization generation, nor any error message. Nothing happended, that’s all I can say so far.

Greetings, AN_IA

Hi AN_IA,

We are currently busy with the preparation of the CADET Workshop. I will come back to you ASAP, in the meantime, it would be great if you could answer the following questions:

  • Have you tried running a single simulation with some mock parameters to make sure your model setup is correct?
  • If so, how long would you expect a single simulation to run?

If the optimizer did not report anything, that strongly hints at an issue in the configuration of your base processes. I would advise you to take a step back and have a look at your simulation first, before optimizing.

Kind regards,
Hannah

Hi,

and thanks for the reply. I can confirm that I had a smoothly running simulation, before I tried to fit any parameters. Simulation times were between 1 and 10 minutes.

In the meantime, I encountered more problems with the original CADET fitting tools, which led me to abandon CADET original fitting and switching to the Optuna fitter. I couldn’t get any CADET fitting to work, because I encountered numerous sqlite3 errors:

•sqlite3.OperationalError: database is locked

•sqlite3.OperationalError: unable to open database file

•sqlite3.OperationalError: disk I/O error

These appeared on both systems that I use. I suppose it has something to do with parallelization. I cannot recall my exact settings (number of cores) at this moment, but I don’t think parallelization was too heavy. I switched to the Optuna fitter, and things are working fine now. Also, I am entering the final phase of my bachelor’s thesis, and unfortunately I don’t have time at the moment to look further into CADET original fitting. Nonetheless, thank you very much for your support. I’m getting by rather well at the moment, using the Optuna fitter.

Kind regards,

AN_IA

2 Likes

Glad you got it to work, all the best in finishing up your thesis!

If you want to look into CADET-Process Optimization tools in the future again, just let us know.

Kind regards,
Hannah

2 Likes

This topic was automatically closed 2 days after the last reply. New replies are no longer allowed.