The tensor charge of the nucleon, which is the first moment of the chiral-odd structure function $h_1$, is studied in the lattice QCD simulation on $12^3 \times 20$ lattice with tadpole improvement. Both the flavor singlet and non-singlet tensor charges are examined numerically with 300 gauge configurations. The results are compared to the axial charge of the nucleon which is the first moment of the chiral-even structure function $g_1$.