Error messages
From Gerris
(Difference between revisions)
| Revision as of 04:28, 17 November 2011 EmilyMLane (Talk | contribs) ← Previous diff |
Revision as of 08:41, 21 November 2011 EmilyMLane (Talk | contribs) Next diff → |
||
| Line 62: | Line 62: | ||
| ---- | ---- | ||
| The same code runs in series, I also tried with 2 and 4 cpus and it failed with the same error message (although at t=1 for 4 nodes and before writing any output for 2 nodes. It seemed to be consistent when it failed. | The same code runs in series, I also tried with 2 and 4 cpus and it failed with the same error message (although at t=1 for 4 nodes and before writing any output for 2 nodes. It seemed to be consistent when it failed. | ||
| + | |||
| + | |||
| + | --- | ||
| + | Gerris Debugging | ||
| + | When running gerris debugging after the fact (i.e. as Stephane suggested doing the following) | ||
| + | % ulimit -c unlimited | ||
| + | % <rerun, crash should display "(core dumped)"> | ||
| + | % ls core* | ||
| + | % gdb gerris3D core | ||
| + | gdb> where | ||
| + | I get: | ||
| + | #0 0x00007fd8198c228e in match_periodic_bc () | ||
| + | from /usr/lib64/libgfs3D-1.3.so.2 | ||
| + | #1 0x00007fd8198902a9 in cell_traverse_boundary_leafs () | ||
| + | from /usr/lib64/libgfs3D-1.3.so.2 | ||
| + | #2 0x00007fd8198902a9 in cell_traverse_boundary_leafs () | ||
| + | from /usr/lib64/libgfs3D-1.3.so.2 | ||
| + | #3 0x00007fd8198902a9 in cell_traverse_boundary_leafs () | ||
| + | from /usr/lib64/libgfs3D-1.3.so.2 | ||
| + | #4 0x00007fd8198902a9 in cell_traverse_boundary_leafs () | ||
| + | from /usr/lib64/libgfs3D-1.3.so.2 | ||
| + | #5 0x00007fd8198902a9 in cell_traverse_boundary_leafs () | ||
| + | from /usr/lib64/libgfs3D-1.3.so.2 | ||
| + | #6 0x00007fd8198902a9 in cell_traverse_boundary_leafs () | ||
| + | from /usr/lib64/libgfs3D-1.3.so.2 | ||
| + | #7 0x00007fd8198c5faf in match_box_bc () from /usr/lib64/libgfs3D-1.3.so.2 | ||
| + | #8 0x00007fd8198c8db3 in domain_foreach () from /usr/lib64/libgfs3D-1.3.so.2 | ||
| + | #9 0x00007fd8198cde21 in gfs_domain_tag_droplets () | ||
| + | from /usr/lib64/libgfs3D-1.3.so.2 | ||
| + | #10 0x00007fd8198ce0ae in gfs_domain_remove_droplets () | ||
| + | from /usr/lib64/libgfs3D-1.3.so.2 | ||
| + | #11 0x00007fd8198d6b59 in gfs_remove_droplets_event () | ||
| + | from /usr/lib64/libgfs3D-1.3.so.2 | ||
| + | #12 0x00007fd8198d2825 in gfs_event_do () from /usr/lib64/libgfs3D-1.3.so.2 | ||
| + | #13 0x00007fd81965fcb7 in slist_container_foreach () | ||
| + | from /usr/lib64/libgts-0.7.so.5 | ||
| + | #14 0x00007fd8198da97f in simulation_run () from /usr/lib64/libgfs3D-1.3.so.2 | ||
| + | #15 0x00007fd8198dc2a8 in gfs_simulation_run () | ||
| + | from /usr/lib64/libgfs3D-1.3.so.2 | ||
| + | #16 0x0000000000402d3f in main () | ||
Revision as of 08:41, 21 November 2011
Emily's Favourite Error Messages
This page is a place for storing error messages so I can compare and contrast.
Nicholson Canyon viscous3D parallel runs
I get the same message on both turbine and the new modelling computer and it occurs at the same position:
Modelling computer:
step: 1 t: 0.50000000 dt: 5.000000e-01 cpu: 31.87250000 real: 34.53414900 MAC projection before after rate niter: 7 residual.bias: -7.007e-03 -5.369e-05 residual.first: 7.016e-03 5.373e-05 2 residual.second: 1.461e-02 7.277e-05 2.1 residual.infty: 3.525e-02 7.272e-04 1.7 Approximate projection niter: 3 residual.bias: -1.151e-04 -5.477e-05 residual.first: 1.547e-04 5.481e-05 1.4 residual.second: 5.344e-04 7.187e-05 2 residual.infty: 2.145e-02 7.902e-04 3 [NIWA-36410:06625] *** Process received signal *** [NIWA-36410:06625] Signal: Segmentation fault (11) [NIWA-36410:06625] Signal code: Address not mapped (1) [NIWA-36410:06625] Failing at address: 0x8 [NIWA-36410:06625] [ 0] /lib64/libpthread.so.0(+0xf2d0) [0x7ffb7880b2d0] [NIWA-36410:06625] [ 1] /usr/lib64/libgfs3D-1.3.so.2(+0x4728e) [0x7ffb794a628e] [NIWA-36410:06625] [ 2] /usr/lib64/libgfs3D-1.3.so.2(+0x152a9) [0x7ffb794742a9] [NIWA-36410:06625] *** End of error message *** mpirun noticed that job rank 0 with PID 6619 on node NIWA-36410 exited on signal 15 (Terminated). 7 additional processes aborted (not shown)
Turbine:
step: 1 t: 0.50000000 dt: 5.000000e-01 cpu: 54.26500000 real: 57.84363200 MAC projection before after rate niter: 7 residual.bias: -7.007e-03 -5.369e-05 residual.first: 7.016e-03 5.373e-05 2 residual.second: 1.461e-02 7.277e-05 2.1 residual.infty: 3.525e-02 7.272e-04 1.7 Approximate projection niter: 3 residual.bias: -1.151e-04 -5.477e-05 residual.first: 1.546e-04 5.481e-05 1.4 residual.second: 5.338e-04 7.187e-05 2 residual.infty: 2.141e-02 7.902e-04 3 [rotor03:20387] *** Process received signal *** [rotor03:20387] Signal: Segmentation fault (11) [rotor03:20387] Signal code: Address not mapped (1) [rotor03:20387] Failing at address: 0x8 [rotor03:20387] [ 0] /lib64/libpthread.so.0 [0x7f64b72c0c00] [rotor03:20387] [ 1] /usr/lib64/libgfs3D-1.3.so.2 [0x7f64b8c0944f] [rotor03:20387] [ 2] /usr/lib64/libgfs3D-1.3.so.2 [0x7f64b8bdc609] [rotor03:20387] *** End of error message *** mpirun noticed that job rank 0 with PID 20381 on node rotor03 exited on signal 15 (Terminated). 7 additional processes aborted (not shown)
The same code runs in series, I also tried with 2 and 4 cpus and it failed with the same error message (although at t=1 for 4 nodes and before writing any output for 2 nodes. It seemed to be consistent when it failed.
---
Gerris Debugging
When running gerris debugging after the fact (i.e. as Stephane suggested doing the following)
% ulimit -c unlimited
% <rerun, crash should display "(core dumped)">
% ls core*
% gdb gerris3D core
gdb> where
I get:
- 0 0x00007fd8198c228e in match_periodic_bc ()
from /usr/lib64/libgfs3D-1.3.so.2
- 1 0x00007fd8198902a9 in cell_traverse_boundary_leafs ()
from /usr/lib64/libgfs3D-1.3.so.2
- 2 0x00007fd8198902a9 in cell_traverse_boundary_leafs ()
from /usr/lib64/libgfs3D-1.3.so.2
- 3 0x00007fd8198902a9 in cell_traverse_boundary_leafs ()
from /usr/lib64/libgfs3D-1.3.so.2
- 4 0x00007fd8198902a9 in cell_traverse_boundary_leafs ()
from /usr/lib64/libgfs3D-1.3.so.2
- 5 0x00007fd8198902a9 in cell_traverse_boundary_leafs ()
from /usr/lib64/libgfs3D-1.3.so.2
- 6 0x00007fd8198902a9 in cell_traverse_boundary_leafs ()
from /usr/lib64/libgfs3D-1.3.so.2
- 7 0x00007fd8198c5faf in match_box_bc () from /usr/lib64/libgfs3D-1.3.so.2
- 8 0x00007fd8198c8db3 in domain_foreach () from /usr/lib64/libgfs3D-1.3.so.2
- 9 0x00007fd8198cde21 in gfs_domain_tag_droplets ()
from /usr/lib64/libgfs3D-1.3.so.2
- 10 0x00007fd8198ce0ae in gfs_domain_remove_droplets ()
from /usr/lib64/libgfs3D-1.3.so.2
- 11 0x00007fd8198d6b59 in gfs_remove_droplets_event ()
from /usr/lib64/libgfs3D-1.3.so.2
- 12 0x00007fd8198d2825 in gfs_event_do () from /usr/lib64/libgfs3D-1.3.so.2
- 13 0x00007fd81965fcb7 in slist_container_foreach ()
from /usr/lib64/libgts-0.7.so.5
- 14 0x00007fd8198da97f in simulation_run () from /usr/lib64/libgfs3D-1.3.so.2
- 15 0x00007fd8198dc2a8 in gfs_simulation_run ()
from /usr/lib64/libgfs3D-1.3.so.2
- 16 0x0000000000402d3f in main ()

