These sounds like there are loose pipes clanging when the water is turned on or off. This is commonly called “water hammer”. Sometimes if the air chamber is blocked or missing this can also cause the condition. If this just started when the water heater was replaced then the problem is likely somewhere in the vicinity of the water heater.
Usually there is an air chamber on both the hot and cold sides of the water heater. This acts like a shock absorber to prevent the pipes from shaking or moving when the water flow is turned on or off. Also, it could be that the pipes are no longer secured in place near the water heater. This also can allow movement when the water is turned on or off. In most cases this does not cause any problems although it is possible for leaks to develop due to the movement in the pipes.
I would start looking for the problem in the area of the water heater water heater. If you don’t find the problem there then expand your search. As always, you should be sure to use a capable contractor to make the needed repairs.