I am having some critical problems over the issue of the kingdom of God and the kingdom of heaven. Please tell me the difference between them.

---

There is no difference between The Kingdom of God and The Kingdom of Heaven.  They are two separate terms for the same kingdom.

"Kingdom of Heaven" occurs 32 times in Matthew's Gospel and no where else in the New Testament.

"Kingdom of God" occurs 66 times in the New Testament, 14 times in  Acts and Paul's Epistles, only 4 times in Matthew, the balance in Luke and Mark.  The four incidents in Matthew are as follows:

Matt.12

[28] But if it is by the Spirit of God that I cast out demons, then the kingdom of God has come upon you.

Matt.19

[24] Again I tell you, it is easier for a camel to go through the eye of a needle than for a rich man to enter the kingdom of God.

Matt.21

[31] Which of the two did the will of his father?" They said, "The first." Jesus said to them, Truly, I say to you, the tax collectors and the harlots go into the kingdom of God before you.
[43] Therefore I tell you, the kingdom of God will be taken away from you and given to a nation producing the fruits of it.

Only Matthew uses the term "kingdom of Heaven."  Mark and Luke use the term "kingdom of God" to relate the exact same utterances of the Lord.  Here is a typical example:

Matt.13

[31] Another parable he put before them, saying, The kingdom of heaven is like a grain of mustard seed which a man took and sowed in his field;

Mark.4 

[30] And he said, With what can we compare the kingdom of God, or what parable shall we use for it?
[31] It is like a grain of mustard seed, which, when sown upon the ground, is the smallest of all the seeds on earth;

Luke.13

[18] He said therefore, What is the kingdom of God like? And to what shall I compare it?
[19] It is like a grain of mustard seed which a man took and sowed in his garden; and it grew and became a tree, and the birds of the air made nests in its branches.

As to why Matthew preferred kingdom of Heaven to kingdom of God, the explanation has long been that Matthew, writing to specifically Jewish readers, inserted "Heaven" for "God" so as not to offend the Jewish sensibilities regarding uttering the name of God or the term that describes Him  This is probably correct, but it leaves us with no explanation for the 4 times Matthew failed to make the switch, or for why he uses "God" in some 53 other instances!

But in any case, if you will compare the synoptic gospels -- Matthew, Mark and Luke, you will find that Matthew's "kingdom of Heaven" exactly parallels Mark's and Luke's "kingdom of God."  So, they are different terms describing the same thing.