Welcome to dpdata’s documentation!
- Overview
- Try dpdata online
- Command line interface
- Supported Formats
- ase/structure format
- ase/traj format
- abacus/lcao/md format
- abacus/pw/md format
- abacus/md format
- abacus/lcao/relax format
- abacus/pw/relax format
- abacus/relax format
- abacus/lcao/scf format
- abacus/pw/scf format
- abacus/scf format
- stru format
- abacus/stru format
- amber/md format
- cp2k/aimd_output format
- cp2k/output format
- dftbplus format
- deepmd/comp format
- deepmd/npy format
- deepmd/hdf5 format
- deepmd/npy/mixed format
- deepmd/raw format
- deepmd format
- fhi_aims/output format
- fhi_aims/md format
- fhi_aims/scf format
- gaussian/gjf format
- gaussian/log format
- gaussian/md format
- gromacs/gro format
- gro format
- lammps/dump format
- dump format
- lammps/lmp format
- lmp format
- list format
- mol_file format
- mol format
- n2p2 format
- openmx/md format
- orca/spout format
- psi4/inp format
- psi4/out format
- pwmat/final.config format
- pwmat/atom.config format
- final.config format
- atom.config format
- pwmat/output format
- pwmat/mlmd format
- pwmat/movement format
- mlmd format
- movement format
- 3dmol format
- pymatgen/computedstructureentry format
- pymatgen/molecule format
- pymatgen/structure format
- qe/pw/scf format
- qe/cp/traj format
- quip/gap/xyz_file format
- quip/gap/xyz format
- sqm/in format
- sqm/out format
- sdf_file format
- sdf format
- siesta/aimd_output format
- siesta/output format
- vasp/outcar format
- outcar format
- vasp/contcar format
- vasp/poscar format
- contcar format
- poscar format
- vasp/string format
- vasp/xml format
- xml format
- xyz format
- Supported Drivers
- Supported Minimizers
- API documentation
- Authors
dpdata
dpdata is a python package for manipulating data formats of software in computational science, including DeePMD-kit, VASP, LAMMPS, GROMACS, Gaussian. dpdata only works with python 3.7 or above. One can download the source code of dpdata by then use This section gives some examples on how dpdata works. Firstly one needs to import the module in a python 3.x compatible code. The typicall workflow of Load data from vasp or lammps or deepmd-kit data files. Manipulate data Dump data to in a desired format or let dpdata infer the format ( The number of atoms, atom types, coordinates are loaded from the The labels provided in the The Software format multi frames labeled class format key vasp poscar False False System ‘vasp/poscar’ vasp outcar True True LabeledSystem ‘vasp/outcar’ vasp xml True True LabeledSystem ‘vasp/xml’ lammps lmp False False System ‘lammps/lmp’ lammps dump True False System ‘lammps/dump’ deepmd raw True False System ‘deepmd/raw’ deepmd npy True False System ‘deepmd/npy’ deepmd raw True True LabeledSystem ‘deepmd/raw’ deepmd npy True True LabeledSystem ‘deepmd/npy’ deepmd npy True True MultiSystems ‘deepmd/npy/mixed’ deepmd npy True False MultiSystems ‘deepmd/npy/mixed’ gaussian log False True LabeledSystem ‘gaussian/log’ gaussian log True True LabeledSystem ‘gaussian/md’ siesta output False True LabeledSystem ‘siesta/output’ siesta aimd_output True True LabeledSystem ‘siesta/aimd_output’ cp2k(deprecated in future) output False True LabeledSystem ‘cp2k/output’ cp2k(deprecated in future) aimd_output True True LabeledSystem ‘cp2k/aimd_output’ cp2k(plug-in) stdout False True LabeledSystem ‘cp2kdata/e_f’ cp2k(plug-in) stdout True True LabeledSystem ‘cp2kdata/md’ QE log False True LabeledSystem ‘qe/pw/scf’ QE log True False System ‘qe/cp/traj’ QE log True True LabeledSystem ‘qe/cp/traj’ Fhi-aims output True True LabeledSystem ‘fhi_aims/md’ Fhi-aims output False True LabeledSystem ‘fhi_aims/scf’ quip/gap xyz True True MultiSystems ‘quip/gap/xyz’ PWmat atom.config False False System ‘pwmat/atom.config’ PWmat movement True True LabeledSystem ‘pwmat/movement’ PWmat OUT.MLMD True True LabeledSystem ‘pwmat/out.mlmd’ Amber multi True True LabeledSystem ‘amber/md’ Amber/sqm sqm.out False False System ‘sqm/out’ Gromacs gro True False System ‘gromacs/gro’ ABACUS STRU False False System ‘abacus/stru’ ABACUS STRU False True LabeledSystem ‘abacus/scf’ ABACUS cif True True LabeledSystem ‘abacus/md’ ABACUS STRU True True LabeledSystem ‘abacus/relax’ ase structure True True MultiSystems ‘ase/structure’ DFTB+ dftbplus False True LabeledSystem ‘dftbplus’ n2p2 n2p2 True True LabeledSystem ‘n2p2’ The Class Use Use For example, for The following commands relating to You may also use the following code to parse muti-system: These properties stored in Available properties are (nframe: number of frames in the system, natoms: total number of atoms in the system) key type dimension are labels description ‘atom_names’ list of str ntypes False The name of each atom type ‘atom_numbs’ list of int ntypes False The number of atoms of each atom type ‘atom_types’ np.ndarray natoms False Array assigning type to each atom ‘cells’ np.ndarray nframes x 3 x 3 False The cell tensor of each frame ‘coords’ np.ndarray nframes x natoms x 3 False The atom coordinates ‘energies’ np.ndarray nframes True The frame energies ‘forces’ np.ndarray nframes x natoms x 3 True The atom forces ‘virials’ np.ndarray nframes x 3 x 3 True The virial tensor of each frame The data stored in The first frames of The last frames of The data stored in Or a simpler command: Frame selection can be implemented by by which only the first and last frames are dumped to dpdata will create a super cell of the current atom configuration. tuple(1,2,3) means don’t copy atom configuration in x direction, make 2 copys in y direction, make 3 copys in z direction. By the following example, each frame of the original system ( By the following example, Random 8 Hf atoms in the system will be replaced by Zr atoms with the atom postion unchanged. A new class In sdf file, all molecules must be of the same topology (i.e. conformers of the same molecular configuration). The low: use medium: before using rdkit, the programm will first assign formal charge of each atom to avoid inappropriate valence exceptions. However, this mode requires the rightness of the bond order information in the given molecule. high: the program will try to fix inappropriate bond orders in aromatic hetreocycles, phosphate, sulfate, carboxyl, nitro, nitrine, guanidine groups. If this procedure fails to sanitize the given molecule, the program will then try to call BondOrderSystem implement a method to assign formal charge for each atom based on the 8-electron rule (see below). Note that it only supports common elements in bio-system: B,C,N,O,P,S,As If a valence of 3 is detected on carbon, the formal charge will be assigned to -1. Because for most cases (in alkynyl anion, isonitrile, cyclopentadienyl anion), the formal charge on 3-valence carbon is -1, and this is also consisent with the 8-electron rule. The format Under this format, systems with the same number of atoms but different formula can be put together for a larger system, especially when the frame numbers in systems are sparse. This also helps to mixture the type information together for model training with type embedding network. Here are examples using dpdata.MultiSystems(*systems).to_deepmd_npy_mixed(“mixed_dir”) One can follow a simple example to add their own format by creating and installing plugins. It’s critical to add the Format class to Installation
git clone https://github.com/deepmodeling/dpdata.git dpdata
pip
to install the module from sourcecd dpdata
pip install .
dpdata
can also by install via pip without sourcepip install dpdata
Quick start
import dpdata
dpdata
isLoad data
d_poscar = dpdata.System("POSCAR", fmt="vasp/poscar")
vasp/poscar
) of the file from the file name extensiond_poscar = dpdata.System("my.POSCAR")
POSCAR
and stored to a data System
called d_poscar
. A data System
(a concept used by deepmd-kit) contains frames that has the same number of atoms of the same type. The order of the atoms should be consistent among the frames in one System
. It is noted that POSCAR
only contains one frame. If the multiple frames stored in, for example, a OUTCAR
is wanted,d_outcar = dpdata.LabeledSystem("OUTCAR")
OUTCAR
, i.e. energies, forces and virials (if any), are loaded by LabeledSystem
. It is noted that the forces of atoms are always assumed to exist. LabeledSystem
is a derived class of System
.System
or LabeledSystem
can be constructed from the following file formats with the format key
in the table passed to argument fmt
:dpdata.MultiSystems
can read data from a dir which may contains many files of different systems, or from single xyz file which contains different systems.dpdata.MultiSystems.from_dir
to read from a directory, dpdata.MultiSystems
will walk in the directory Recursively and find all file with specific file_name. Supports all the file formats that dpdata.LabeledSystem
supports.dpdata.MultiSystems.from_file
to read from single file. Single-file support is available for the quip/gap/xyz
and ase/structure
formats.quip/gap xyz
files, single .xyz file may contain many different configurations with different atom numbers and atom type.Class dpdata.MultiSystems
may be useful.# load data
xyz_multi_systems = dpdata.MultiSystems.from_file(
file_name="tests/xyz/xyz_unittest.xyz", fmt="quip/gap/xyz"
)
vasp_multi_systems = dpdata.MultiSystems.from_dir(
dir_name="./mgal_outcar", file_name="OUTCAR", fmt="vasp/outcar"
)
# use wildcard
vasp_multi_systems = dpdata.MultiSystems.from_dir(
dir_name="./mgal_outcar", file_name="*OUTCAR", fmt="vasp/outcar"
)
# print the multi_system infomation
print(xyz_multi_systems)
print(xyz_multi_systems.systems) # return a dictionaries
# print the system infomation
print(xyz_multi_systems.systems["B1C9"].data)
# dump a system's data to ./my_work_dir/B1C9_raw folder
xyz_multi_systems.systems["B1C9"].to_deepmd_raw("./my_work_dir/B1C9_raw")
# dump all systems
xyz_multi_systems.to_deepmd_raw("./my_deepmd_data/")
from dpdata import LabeledSystem, MultiSystems
from glob import glob
"""
process multi systems
"""
fs = glob("./*/OUTCAR") # remeber to change here !!!
ms = MultiSystems()
for f in fs:
try:
ls = LabeledSystem(f)
except:
print(f)
if len(ls) > 0:
ms.append(ls)
ms.to_deepmd_raw("deepmd")
ms.to_deepmd_npy("deepmd")
Access data
System
and LabeledSystem
can be accessed by operator []
with the key of the property supplied, for examplecoords = d_outcar["coords"]
Dump data
System
or LabeledSystem
can be dumped in ‘lammps/lmp’ or ‘vasp/poscar’ format, for example:d_outcar.to("lammps/lmp", "conf.lmp", frame_idx=0)
d_outcar
will be dumped to ‘conf.lmp’d_outcar.to("vasp/poscar", "POSCAR", frame_idx=-1)
d_outcar
will be dumped to ‘POSCAR’.LabeledSystem
can be dumped to deepmd-kit raw format, for exampled_outcar.to("deepmd/raw", "dpmd_raw")
dpdata.LabeledSystem("OUTCAR").to("deepmd/raw", "dpmd_raw")
dpdata.LabeledSystem("OUTCAR").sub_system([0, -1]).to("deepmd/raw", "dpmd_raw")
dpmd_raw
.replicate
dpdata.System("./POSCAR").replicate(
(
1,
2,
3,
)
)
perturb
dpdata.System('./POSCAR')
) is perturbed to generate three new frames. For each frame, the cell is perturbed by 5% and the atom positions are perturbed by 0.6 Angstrom. atom_pert_style
indicates that the perturbation to the atom positions is subject to normal distribution. Other available options to atom_pert_style
areuniform
(uniform in a ball), and const
(uniform on a sphere).perturbed_system = dpdata.System("./POSCAR").perturb(
pert_num=3,
cell_pert_fraction=0.05,
atom_pert_distance=0.6,
atom_pert_style="normal",
)
print(perturbed_system.data)
replace
s = dpdata.System("tests/poscars/POSCAR.P42nmc", fmt="vasp/poscar")
s.replace("Hf", "Zr", 8)
s.to_vasp_poscar("POSCAR.P42nmc.replace")
BondOrderSystem
BondOrderSystem
which inherits from class System
is introduced in dpdata. This new class contains information of chemical bonds and formal charges (stored in BondOrderSystem.data['bonds']
, BondOrderSystem.data['formal_charges']
). Now BondOrderSystem can only read from .mol/.sdf formats, because of its dependency on rdkit (which means rdkit must be installed if you want to use this function). Other formats, such as pdb, must be converted to .mol/.sdf format (maybe with software like open babel).import dpdata
system_1 = dpdata.BondOrderSystem(
"tests/bond_order/CH3OH.mol", fmt="mol"
) # read from .mol file
system_2 = dpdata.BondOrderSystem(
"tests/bond_order/methane.sdf", fmt="sdf"
) # read from .sdf file
BondOrderSystem
also supports initialize from a rdkit.Chem.rdchem.Mol
object directly.from rdkit import Chem
from rdkit.Chem import AllChem
import dpdata
mol = Chem.MolFromSmiles("CC")
mol = Chem.AddHs(mol)
AllChem.EmbedMultipleConfs(mol, 10)
system = dpdata.BondOrderSystem(rdkit_mol=mol)
Bond Order Assignment
BondOrderSystem
implements a more robust sanitize procedure for rdkit Mol, as defined in dpdata.rdkit.santizie.Sanitizer
. This class defines 3 level of sanitization process by: low, medium and high. (default is medium).rdkit.Chem.SanitizeMol()
function to sanitize molecule.obabel
to pre-process the mol and repeat the sanitization procedure. That is to say, if you wan’t to use this level of sanitization, please ensure ``obabel`` is installed in the environment. According to our test, our sanitization procedure can successfully read 4852 small molecules in the PDBBind-refined-set. It is necessary to point out that the in the molecule file (mol/sdf), the number of explicit hydrogens has to be correct. Thus, we recommend to use obabel xxx -O xxx -h
to pre-process the file. The reason why we do not implement this hydrogen-adding procedure in dpdata is that we can not ensure its correctness.import dpdata
for sdf_file in glob.glob("bond_order/refined-set-ligands/obabel/*sdf"):
syst = dpdata.BondOrderSystem(sdf_file, sanitize_level="high", verbose=False)
Formal Charge Assignment
import dpdata
syst = dpdata.BondOrderSystem("tests/bond_order/CH3NH3+.mol", fmt="mol")
print(syst.get_formal_charges()) # return the formal charge on each atom
print(syst.get_charge()) # return the total charge of the system
Mixed Type Format
deepmd/npy/mixed
is the mixed type numpy format for DeePMD-kit, and can be loaded or dumped through class dpdata.MultiSystems
.deepmd/npy/mixed
format:- Load a mixed type data into a MultiSystems:
```python
import dpdata
dpdata.MultiSystems().load_systems_from_file("mixed_dir", fmt="deepmd/npy/mixed")
Plugins
entry_points['dpdata.plugins']
in ``pyproject.toml` <plugin_example/pyproject.toml>`_:[project.entry-points.'dpdata.plugins']
random = "dpdata_random:RandomFormat"