JERUSALEM, June 4— Prime Minister Yitzhak Shamir appeared to rebuff President Mikhail S. Gorbachev today by saying Israel will not dictate to Jewish immigrants from the Soviet Union where they can live. 
Mr. Shamir's comments did not contradict assertions by other senior Israeli officials on Sunday and today that only a few hundred immigrants from among the tens of thousands arriving here from the Soviet Union had chosen to live in settlements in the Israeli-occupied West Bank and Gaza. But he seemed to leave the door open for those who wished to do so. 
Mr. Gorbachev said on Sunday that the Soviet Union might reconsider granting exit visas to Jews unless it received assurances from Israel that no Jewish immigrants would settle in the occupied territories. 
Choice of Where to Live
Speaking to a group of Israelis today, Mr. Shamir said, ''If the Soviet Union or President Gorbachev does not think it can today tell its citizens where to live, it is as clear as the sun that we, as followers of freedom and democracy, cannot limit this category.''
He went on to say that Israel would not tell its citizens where they could live ''in the land of Israel.''
The Israeli Government says that only a handful of Jewish immigrants from the Soviet Union have settled in the occupied territories. Western diplomats say that if new settlements in East Jerusalem are included, the number rises to as many as 10 percent of Soviet Jewish immigrants. 
The last monthly figure for Soviet Jewish immigration, from May, is 10,200. In April the figure was 10,600. 
Negotiating a Coalition
Mr. Shamir seemed close today to forming a Government next week based on a coalition of rightist and religious parties.
He said he would inform President Chaim Herzog by Thursday that he could form a government and put it to a confidence vote in Parliament ''at the beginning of next week,'' thus ending a period of more than two months during which Israel has drifted under an interim government. His failure to form a government by Thursday or to win the confidence vote could open the door for new elections.
Although Mr. Shamir did not specify which parties would be included in his government, officials and analysts said it would be a narrow coalition made up of his Likud party in alliance with right-wing and religious parties.
Parliament dissolved the Government on March 15 because of Mr. Shamir's refusal to accept American terms for holding Israeli-Palestinian talks. If the new government is exclusively rightist, it is almost certain to be hostile to the proposal by Secretary of State James A. Baker 3d calling for elections in the West Bank and Gaza to choose a Palestinian leadership that would negotiate with Israel on an end to the Israeli-Palestinian conflict.
U.S. 'Not Sure' of Soviet Stand
WASHINGTON, June 4 (AP) - The State Department spokeswoman, Margaret Tutwiler, said today, ''We're not sure what President Gorbachev meant by his comments.''
''Nothing that was said during the private summit discussions indicated in any way that the Soviets won't live up to their commitments to permit Soviet Jewish emigration,'' she said. ''We expect that they will continue to do so. They know how very important this is to us.''
Group Is Critical of Gorbachev
The National Conference on Soviet Jewry declared yesterday that the imposition of Soviet restrictions on Jewish emigration unless Israel barred resettlement in the occupied territories would be ''an unacceptable bowing to outside pressure.'' It also would be a false issue, the group asserted, since only ''a minuscule number'' of emigres have gone there and since they have done so on their own rather than in line with Israeli policy.
In a telegram to President Gorbachev, the National Conference said curtailment of emigration would cause it to reconsider its stand on the lifting of trade restrictions.