Expected behavior.
Hello everyone,
I have tried to implement parameter fitting into my model. I get this error message:
(AttributeError: 'NoneType' object has no attribute '_get_values_from_container').
Expected behavior:
After entering the optimization, CADET should optimize and start printing out the results of each generation.
Actual behavior
The optimization seems to start. For a very long time, nothing happens. Then, before anything is printed out, the given error message appears, and the execution fails.
Here is the complete, very long error message:
_RemoteTraceback Traceback (most recent call last)
_RemoteTraceback:
“”"
Traceback (most recent call last):
File “e:\Programme\Miniforge3\envs\cadet_workshop\Lib\site-packages\joblib\externals\loky\process_executor.py”, line 490, in _process_worker
r = call_item()
^^^^^^^^^^^
File “e:\Programme\Miniforge3\envs\cadet_workshop\Lib\site-packages\joblib\externals\loky\process_executor.py”, line 291, in call
return self.fn(*self.args, **self.kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “e:\Programme\Miniforge3\envs\cadet_workshop\Lib\site-packages\joblib\parallel.py”, line 607, in call
return [func(*args, **kwargs) for func, args, kwargs in self.items]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “e:\Programme\Miniforge3\envs\cadet_workshop\Lib\site-packages\joblib\parallel.py”, line 607, in
return [func(*args, **kwargs) for func, args, kwargs in self.items]
^^^^^^^^^^^^^^^^^^^^^
File “e:\Programme\Miniforge3\envs\cadet_workshop\Lib\site-packages\CADETProcess\optimization\optimizationProblem.py”, line 792, in target_wrapper
results = self._evaluate_individual(
^^^^^^^^^^^^^^^^^^^^^^^^^^
File “e:\Programme\Miniforge3\envs\cadet_workshop\Lib\site-packages\CADETProcess\optimization\optimizationProblem.py”, line 838, in _evaluate_individual
value = self._evaluate(x, eval_fun, force)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “e:\Programme\Miniforge3\envs\cadet_workshop\Lib\site-packages\CADETProcess\optimization\optimizationProblem.py”, line 925, in _evaluate
result = step.evaluate(current_request)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “e:\Programme\Miniforge3\envs\cadet_workshop\Lib\site-packages\CADETProcess\optimization\optimizationProblem.py”, line 4123, in call
results = self.evaluator(request, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “e:\Programme\Miniforge3\envs\cadet_workshop\Lib\site-packages\CADETProcess\simulator\simulator.py”, line 275, in simulate
results = self.simulate_n_cycles(
^^^^^^^^^^^^^^^^^^^^^^^
File “e:\Programme\Miniforge3\envs\cadet_workshop\Lib\site-packages\CADETProcess\log.py”, line 164, in wrapper
result = function(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^
File “e:\Programme\Miniforge3\envs\cadet_workshop\Lib\site-packages\CADETProcess\log.py”, line 225, in wrapper
results = function(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^
File “e:\Programme\Miniforge3\envs\cadet_workshop\Lib\site-packages\CADETProcess\log.py”, line 200, in wrapper
raise e
File “e:\Programme\Miniforge3\envs\cadet_workshop\Lib\site-packages\CADETProcess\log.py”, line 192, in wrapper
return function(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^
File “e:\Programme\Miniforge3\envs\cadet_workshop\Lib\site-packages\CADETProcess\simulator\simulator.py”, line 334, in simulate_n_cycles
return self._run(process, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “e:\Programme\Miniforge3\envs\cadet_workshop\Lib\site-packages\CADETProcess\simulator\cadetAdapter.py”, line 154, in wrapper
results = func(self, process, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “e:\Programme\Miniforge3\envs\cadet_workshop\Lib\site-packages\CADETProcess\simulator\cadetAdapter.py”, line 290, in _run
f"Simulation of {process.name} with parameters {process.config} failed."
File “e:\Programme\Miniforge3\envs\cadet_workshop\Lib\site-packages\CADETProcess\dataStructure\aggregator.py”, line 88, in repr
return f"NumpyProxyA{self._get_values_from_aggregator().repr()[1:]}"
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “e:\Programme\Miniforge3\envs\cadet_workshop\Lib\site-packages\CADETProcess\dataStructure\aggregator.py”, line 46, in _get_values_from_aggregator
return self.aggregator._get_values_from_container(self.instance, transpose=True, check=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: ‘NoneType’ object has no attribute ‘_get_values_from_container’
“”"
The above exception was the direct cause of the following exception:
AttributeError Traceback (most recent call last)
Cell In[62], line 3
1 #Optimieren
----> 3 optimization_results = optimizer.optimize(
4 optimization_problem,
5 use_checkpoint=False
6 )
File e:\Programme\Miniforge3\envs\cadet_workshop\Lib\site-packages\CADETProcess\optimization\optimizer.py:277, in OptimizerBase.optimize(self, optimization_problem, x0, save_results, results_directory, use_checkpoint, overwrite_results_directory, exist_ok, log_level, reinit_cache, delete_cache, *args, **kwargs)
274 plt.switch_backend(“agg”)
276 start = time.time()
→ 277 self._run(self.optimization_problem, x0, *args, **kwargs)
278 time_elapsed = time.time() - start
280 self.results.time_elapsed = time_elapsed
File e:\Programme\Miniforge3\envs\cadet_workshop\Lib\site-packages\CADETProcess\optimization\pymooAdapter.py:165, in PymooInterface._run(self, optimization_problem, x0)
162 X = pop.get(“X”).tolist()
164 # Evaluate objectives and report results
→ 165 algorithm.evaluator.eval(problem, pop)
167 F = pop.get(“F”).tolist()
168 if optimization_problem.n_nonlinear_constraints > 0:
File e:\Programme\Miniforge3\envs\cadet_workshop\Lib\site-packages\pymoo\core\evaluator.py:69, in Evaluator.eval(self, problem, pop, skip_already_evaluated, evaluate_values_of, count_evals, **kwargs)
65 # evaluate the solutions (if there are any)
66 if len(I) > 0:
67
68 # do the actual evaluation - call the sub-function to set the corresponding values to the population
—> 69 self._eval(problem, pop[I], evaluate_values_of, **kwargs)
71 # update the function evaluation counter
72 if count_evals:
File e:\Programme\Miniforge3\envs\cadet_workshop\Lib\site-packages\pymoo\core\evaluator.py:90, in Evaluator._eval(self, problem, pop, evaluate_values_of, **kwargs)
87 X = pop.get(“X”)
89 # call the problem to evaluate the solutions
—> 90 out = problem.evaluate(X, return_values_of=evaluate_values_of, return_as_dictionary=True, **kwargs)
92 # for each of the attributes set it to the problem
93 for key, val in out.items():
File e:\Programme\Miniforge3\envs\cadet_workshop\Lib\site-packages\pymoo\core\problem.py:257, in Problem.evaluate(self, X, return_values_of, return_as_dictionary, *args, **kwargs)
254 only_single_value = not (isinstance(X, list) or isinstance(X, np.ndarray))
256 # this is where the actual evaluation takes place
→ 257 _out = self.do(X, return_values_of, *args, **kwargs)
259 out = {}
260 for k, v in _out.items():
261
262 # copy it to a numpy array (it might be one of jax at this point)
File e:\Programme\Miniforge3\envs\cadet_workshop\Lib\site-packages\pymoo\core\problem.py:299, in Problem.do(self, X, return_values_of, *args, **kwargs)
297 self._evaluate_elementwise(X, out, *args, **kwargs)
298 else:
→ 299 self._evaluate_vectorized(X, out, *args, **kwargs)
301 # finally format the output dictionary
302 out = self._format_dict(out, len(X), return_values_of)
File e:\Programme\Miniforge3\envs\cadet_workshop\Lib\site-packages\pymoo\core\problem.py:307, in Problem._evaluate_vectorized(self, X, out, *args, **kwargs)
306 def _evaluate_vectorized(self, X, out, *args, **kwargs):
→ 307 self._evaluate(X, out, *args, **kwargs)
File e:\Programme\Miniforge3\envs\cadet_workshop\Lib\site-packages\CADETProcess\optimization\pymooAdapter.py:293, in PymooProblem._evaluate(self, X, out, *args, **kwargs)
291 opt = self.optimization_problem
292 if opt.n_objectives > 0:
→ 293 F = opt.evaluate_objectives(
294 X,
295 untransform=True,
296 get_dependent_values=True,
297 ensure_minimization=True,
298 parallelization_backend=self.parallelization_backend,
299 )
300 out[“F”] = np.array(F)
302 if opt.n_nonlinear_constraints > 0:
File e:\Programme\Miniforge3\envs\cadet_workshop\Lib\site-packages\CADETProcess\optimization\optimizationProblem.py:206, in OptimizationProblem.ensures2d..wrapper_ensures2d(self, X, *args, **kwargs)
203 X_2d = np.array(X, ndmin=2)
205 # Call and ensure results are 2D
→ 206 Y = func(self, X_2d, *args, **kwargs)
207 Y_2d = Y.reshape((len(X_2d), -1))
209 # Reshape back to original length of X
File e:\Programme\Miniforge3\envs\cadet_workshop\Lib\site-packages\CADETProcess\optimization\optimizationProblem.py:169, in OptimizationProblem.untransforms..wrapper_untransforms(self, x, untransform, *args, **kwargs)
166 if untransform:
167 x = self.untransform(x)
→ 169 return func(self, x, *args, **kwargs)
File e:\Programme\Miniforge3\envs\cadet_workshop\Lib\site-packages\CADETProcess\optimization\optimizationProblem.py:187, in OptimizationProblem.gets_dependent_values..wrapper_gets_dependent_values(self, x, get_dependent_values, *args, **kwargs)
184 if get_dependent_values:
185 x = self.get_dependent_values(x)
→ 187 return func(self, x, *args, **kwargs)
File e:\Programme\Miniforge3\envs\cadet_workshop\Lib\site-packages\CADETProcess\optimization\optimizationProblem.py:228, in OptimizationProblem.ensures_minimization..wrap..wrapper_ensures_minimization(self, ensure_minimization, *args, **kwargs)
221 @wraps(func)
222 def wrapper_ensures_minimization(
223 self: OptimizationProblem,
(…) 226 **kwargs: Any,
227 ) → Any:
→ 228 s = func(self, *args, **kwargs)
230 if ensure_minimization:
231 s = self.transform_maximization(s, scores)
File e:\Programme\Miniforge3\envs\cadet_workshop\Lib\site-packages\CADETProcess\optimization\optimizationProblem.py:1171, in OptimizationProblem.evaluate_objectives(self, X, parallelization_backend, force)
1136 @ensures2d
1137 @untransforms
1138 @gets_dependent_values
(…) 1144 force: bool = False,
1145 ) → np.ndarray:
1146 “”"
1147 Evaluate objective functions for each individual x in population X.
1148
(…) 1169 _evaluate
1170 “”"
→ 1171 return self._evaluate_population(
1172 target_functions=self.objectives,
1173 X=X,
1174 parallelization_backend=parallelization_backend,
1175 force=force,
1176 )
File e:\Programme\Miniforge3\envs\cadet_workshop\Lib\site-packages\CADETProcess\optimization\optimizationProblem.py:801, in OptimizationProblem._evaluate_population(self, target_functions, X, parallelization_backend, force)
797 self.cache.close()
799 return results
→ 801 results = parallelization_backend.evaluate(target_wrapper, X)
802 return np.array(results, ndmin=2)
File e:\Programme\Miniforge3\envs\cadet_workshop\Lib\site-packages\CADETProcess\optimization\parallelizationBackend.py:153, in Joblib.evaluate(self, function, population)
137 “”"
138 Evaluate the function in parallalel for all individuals of the population.
139
(…) 150 List of results of function evaluations.
151 “”"
152 backend = Parallel(n_jobs=self.n_cores, verbose=self.verbose)
→ 153 results = backend(delayed(function)(x) for x in population)
155 return results
File e:\Programme\Miniforge3\envs\cadet_workshop\Lib\site-packages\joblib\parallel.py:2072, in Parallel.call(self, iterable)
2066 # The first item from the output is blank, but it makes the interpreter
2067 # progress until it enters the Try/Except block of the generator and
2068 # reaches the first yield statement. This starts the asynchronous
2069 # dispatch of the tasks to the workers.
2070 next(output)
→ 2072 return output if self.return_generator else list(output)
File e:\Programme\Miniforge3\envs\cadet_workshop\Lib\site-packages\joblib\parallel.py:1682, in Parallel._get_outputs(self, iterator, pre_dispatch)
1679 yield
1681 with self._backend.retrieval_context():
→ 1682 yield from self._retrieve()
1684 except GeneratorExit:
1685 # The generator has been garbage collected before being fully
1686 # consumed. This aborts the remaining tasks if possible and warn
1687 # the user if necessary.
1688 self._exception = True
File e:\Programme\Miniforge3\envs\cadet_workshop\Lib\site-packages\joblib\parallel.py:1784, in Parallel._retrieve(self)
1778 while self._wait_retrieval():
1779 # If the callback thread of a worker has signaled that its task
1780 # triggered an exception, or if the retrieval loop has raised an
1781 # exception (e.g. GeneratorExit), exit the loop and surface the
1782 # worker traceback.
1783 if self._aborting:
→ 1784 self._raise_error_fast()
1785 break
1787 nb_jobs = len(self._jobs)
File e:\Programme\Miniforge3\envs\cadet_workshop\Lib\site-packages\joblib\parallel.py:1859, in Parallel._raise_error_fast(self)
1855 # If this error job exists, immediately raise the error by
1856 # calling get_result. This job might not exists if abort has been
1857 # called directly or if the generator is gc’ed.
1858 if error_job is not None:
→ 1859 error_job.get_result(self.timeout)
File e:\Programme\Miniforge3\envs\cadet_workshop\Lib\site-packages\joblib\parallel.py:758, in BatchCompletionCallBack.get_result(self, timeout)
752 backend = self.parallel._backend
754 if backend.supports_retrieve_callback:
755 # We assume that the result has already been retrieved by the
756 # callback thread, and is stored internally. It’s just waiting to
757 # be returned.
→ 758 return self._return_or_raise()
760 # For other backends, the main thread needs to run the retrieval step.
761 try:
File e:\Programme\Miniforge3\envs\cadet_workshop\Lib\site-packages\joblib\parallel.py:773, in BatchCompletionCallBack._return_or_raise(self)
771 try:
772 if self.status == TASK_ERROR:
→ 773 raise self._result
774 return self._result
775 finally:
AttributeError: ‘NoneType’ object has no attribute ‘_get_values_from_container’
How to produce bug (including a minimal reproducible example)
The problem occurs, when I try to fit the film diffusion for every single one of my 6 components simultaneously. I’ve tried the same model, but then excluded all comonents except the one with index 5 for film diffusion fitting. This worked.
I provide an MRE: Fit_Error_get_result_MRE.py. It requires IA-values_pH8_MRE.xlsx for the reference data. Both in the provided repository.
I’ve started to speculate that this error might be caused by at least one of the evaluation-simulations failing. Maybe you could confirm this, or else explain, what is actually happening, and eventually, how it could be fixed? That would be much appreciated.
Greetings, AN_IA