[BUG] _dpgen_auto_test post error using deepmd for relaxation

[ZHANG-JINYU-1994]

Summary

When I am running "dpgen autotest post relaxation.json" to test dp potential, the error below occurs. The attachment 1 is the related files in relax_task folder. As we can see from log.lammps and dump.relax, 7 configurations at timestep 0,100,107,174,274,374,391 are dumped, but the error says there is only 5 energy values. I checked other successful results, the number of configurations is <= 4 (e.g. attachment2). So far no similar error occurs when I am running autotest for dft calculation.

File "/home/jzhang/anaconda3/envs/deepmd/bin/dpgen", line 8, in
sys.exit(main())
File "/home/jzhang/anaconda3/envs/deepmd/lib/python3.10/site-packages/dpgen/main.py", line 185, in mai
args.func(args)
File "/home/jzhang/anaconda3/envs/deepmd/lib/python3.10/site-packages/dpgen/auto_test/run.py", line 57
run_task(args.TASK, args.PARAM, args.MACHINE)
File "/home/jzhang/anaconda3/envs/deepmd/lib/python3.10/site-packages/dpgen/auto_test/run.py", line 44
post_equi(confs, inter_parameter)
File "/home/jzhang/anaconda3/envs/deepmd/lib/python3.10/site-packages/dpgen/auto_test/common_equi.py",ost_equi
res = inter.compute(ii)
File "/home/jzhang/anaconda3/envs/deepmd/lib/python3.10/site-packages/dpgen/auto_test/Lammps.py", linee
d_dump = loadfn(contcar)
File "/home/jzhang/anaconda3/envs/deepmd/lib/python3.10/site-packages/monty/serialization.py", line 72
return json.load(fp, *args, **kwargs)
File "/home/jzhang/anaconda3/envs/deepmd/lib/python3.10/json/init.py", line 293, in load
return loads(fp.read(),
File "/home/jzhang/anaconda3/envs/deepmd/lib/python3.10/json/init.py", line 359, in loads
return cls(**kw).decode(s)
File "/home/jzhang/anaconda3/envs/deepmd/lib/python3.10/site-packages/monty/json.py", line 475, in dec
return self.process_decoded(d)
File "/home/jzhang/anaconda3/envs/deepmd/lib/python3.10/site-packages/monty/json.py", line 427, in pro
return cls.from_dict(data)
File "/home/jzhang/anaconda3/envs/deepmd/lib/python3.10/site-packages/monty/json.py", line 191, in fro
return cls(**decoded)
File "/home/jzhang/anaconda3/envs/deepmd/lib/python3.10/site-packages/dpdata/system.py", line 261, in
self.check_data()
File "/home/jzhang/anaconda3/envs/deepmd/lib/python3.10/site-packages/dpdata/system.py", line 281, in
dd.check(self)
File "/home/jzhang/anaconda3/envs/deepmd/lib/python3.10/site-packages/dpdata/system.py", line 121, in
raise DataError("Shape of %s is %s, but expected %s" % (self.name,
dpdata.system.DataError: Shape of energies is (5,), but expected (7,)_

DPGEN Version and Platform

DPGEN 0.11.0
Linux version 5.10.0-11-amd64 ([email protected]) (gcc-10 (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.2) #1 SMP Debian 5.10.92-1 (2022-01-18)

LAMMPS 23 Jun 2022 - Update 1, Deepmd-kit v2.1.5

Similar results also happen when I am using Bohrium.

Job submission and computing cluster configuration

Expected Behavior

Actual Behavior

Steps to Reproduce

Please do the test using attachment 1 & 3 and run "dpgen autotest post relax_dp.json"

Further Information, Files, and Links

attachment1.zip
attachment2.zip
relax_dp.json.zip

[ZHANG-JINYU-1994]

I found the error is very similar to issue #1051

[ZHANG-JINYU-1994]

I think that the error is related to the dumped timesteps interrupted by the three minimizations in in.lammps. The number of configurations (steps 0,100,107,174,274,374,391) in dump.relax is not the same as that expected by autotest (steps 0,100,200,300,391). One solution to avoid this is to remove two of the minimization manually by 【sed -i '18,21d' ./confs/mp-*/relaxation/in.lammps】

[kevinwenminion]

Hi Jinyu. In the autotest code, we first read 'dump.relax' file and the dumped timesteps are 0, 100, 107, 174, 274, 374, 391. Then we will look for the energy, forces, and stresses for these time steps in 'log.lammps'. However, there are only 0, 100, 107, 174, 200, 300, and 391 timesteps in 'log.lammps'. In this case, the energy, forces, and stresses for time steps 274, 374 are missing. This seems to be an inconsistency between 'log.lammps' and 'dump.relax' introduced by the new version of lammps. We will find a way to consider this 'inconsistency' in autotest.

[ZHANG-JINYU-1994]

Thanks for your reply. For now I just leave only one minimization which is OK for my calculation.

[LJyanyan]

Hi Jinyu. In the autotest code, we first read 'dump.relax' file and the dumped timesteps are 0, 100, 107, 174, 274, 374, 391. Then we will look for the energy, forces, and stresses for these time steps in 'log.lammps'. However, there are only 0, 100, 107, 174, 200, 300, and 391 timesteps in 'log.lammps'. In this case, the energy, forces, and stresses for time steps 274, 374 are missing. This seems to be an inconsistency between 'log.lammps' and 'dump.relax' introduced by the new version of lammps. We will find a way to consider this 'inconsistency' in autotest.

Hello, I also reported the same problem, can you provide me with some solutions?

[ZHANG-JINYU-1994]

As I said above, just delete two of the three minimization steps will work.

[LJyanyan]

I also tried to delete it, but still reported the same problem.This is my in.lammps script, can you help me with some suggestions.
clear
units metal
dimension 3
boundary p p p
atom_style atomic
box tilt large
read_data conf.lmp
mass 1 63.546
mass 2 118.710
neigh_modify every 1 delay 0 check no
pair_style deepmd frozen_model.pb
pair_coeff * *
compute mype all pe
thermo 100
thermo_style custom step pe pxx pyy pzz pxy pxz pyz lx ly lz vol c_mype
dump 1 all custom 100 dump.relax id type xs ys zs fx fy fz
min_style cg
fix 1 all box/relax iso 0.0
minimize 1.000000e-12 1.000000e-06 5000 500000
fix 1 all box/relax aniso 0.0
fix 1 all box/relax tri 0.0
variable N equal count(all)
variable V equal vol
variable E equal "c_mype"
variable tmplx equal lx
variable tmply equal ly
variable Pxx equal pxx
variable Pyy equal pyy
variable Pzz equal pzz
variable Pxy equal pxy
variable Pxz equal pxz
variable Pyz equal pyz
variable Epa equal ${E}/${N}
variable Vpa equal ${V}/${N}
variable AA equal (${tmplx}*${tmply})
print "All done"
print "Total number of atoms = ${N}"
print "Final energy per atoms = ${Epa}"
print "Final volume per atoms = ${Vpa}"
print "Final Base area = ${AA}"
print "Final Stress (xx yy zz xy xz yz) = ${Pxx} ${Pyy} ${Pzz} ${Pxy} ${Pxz} ${Pyz}"

[ZHANG-JINYU-1994]

Here is my code:
clear
units metal
dimension 3
boundary p p p
atom_style atomic
box tilt large
read_data conf.lmp
mass 1 15.999
mass 2 91.224
neigh_modify every 1 delay 0 check no
pair_style deepmd OZrCe_se_1_compress.pb
pair_coeff * *
compute mype all pe
thermo 100
thermo_style custom step pe pxx pyy pzz pxy pxz pyz lx ly lz vol c_mype
dump 1 all custom 100 dump.relax id type xs ys zs fx fy fz
min_style cg
fix 1 all box/relax tri 0.0
minimize 0.000000e+00 1.000000e-10 5000 500000
variable N equal count(all)
variable V equal vol
variable E equal "c_mype"
variable tmplx equal lx
variable tmply equal ly
variable Pxx equal pxx
variable Pyy equal pyy
variable Pzz equal pzz
variable Pxy equal pxy
variable Pxz equal pxz
variable Pyz equal pyz
variable Epa equal ${E}/${N}
variable Vpa equal ${V}/${N}
variable AA equal (${tmplx}*${tmply})
print "All done"
print "Total number of atoms = ${N}"
print "Final energy per atoms = ${Epa}"
print "Final volume per atoms = ${Vpa}"
print "Final Base area = ${AA}"
print "Final Stress (xx yy zz xy xz yz) = ${Pxx} ${Pyy} ${Pzz} ${Pxy} ${Pxz} ${Pyz}"

[LJyanyan]

Here is my code: clear units metal dimension 3 boundary p p p atom_style atomic box tilt large read_data conf.lmp mass 1 15.999 mass 2 91.224 neigh_modify every 1 delay 0 check no pair_style deepmd OZrCe_se_1_compress.pb pair_coeff * * compute mype all pe thermo 100 thermo_style custom step pe pxx pyy pzz pxy pxz pyz lx ly lz vol c_mype dump 1 all custom 100 dump.relax id type xs ys zs fx fy fz min_style cg fix 1 all box/relax tri 0.0 minimize 0.000000e+00 1.000000e-10 5000 500000 variable N equal count(all) variable V equal vol variable E equal "c_mype" variable tmplx equal lx variable tmply equal ly variable Pxx equal pxx variable Pyy equal pyy variable Pzz equal pzz variable Pxy equal pxy variable Pxz equal pxz variable Pyz equal pyz variable Epa equal ${E}/${N} variable Vpa equal ${V}/${N} variable AA equal (${tmplx}*${tmply}) print "All done" print "Total number of atoms = ${N}" print "Final energy per atoms = ${Epa}" print "Final volume per atoms = ${Vpa}" print "Final Base area = ${AA}" print "Final Stress (xx yy zz xy xz yz) = ${Pxx} ${Pyy} ${Pzz} ${Pxy} ${Pxz} ${Pyz}"

Thank you very much for the script, I tried to use the minimization method you used, and still reported the same problem. Can you still provide me with ideas for solutions?

[Vibsteamer]

@LJyanyan @ZHANG-JINYU-1994
Dear users,

Here is a workaround, please set the dump frequency as large as possible, e.g. 999999, to let both log and dump.relax/other_files only dump at the first and the last frame of each fix. This will force them to be consistent and skip this annoying error, and you can use multiple fixes as you want.

Of cours the itermediate frames will be omitted by this workaround, hoping they are not important.
Also please make sure the relaxation could be finished normally, ortherwise, it will just keep running for ten thousands of years without any outputs as the reply to your aching concern.

e.g.:
dump 1 all custom 100 dump.relax id type xs ys zs fx fy fz
--->
dump 1 all custom 999999 dump.relax id type xs ys zs fx fy fz

Best wishes to your endeavor and life ~