An ex-clerk to the Commons and a former electoral commissioner said there was a degree of "public confusion" over the process and questioned its fairness.
Brecon and Radnorshire MP Chris Davies pleaded guilty to two charges of making a false expenses claim in March.
The Petition Officer said arrangements fully complied with legislation.
The petition has been opened and voters in the constituency will be able to sign the petition until 20 June in six different locations in the area.
Under the rules a maximum of 10 stations can be chosen within a constituency for signing in person.
A by-election will be held if 10% of the electorate - 5,303 voters in this case - sign.
Sir Paul Silk, an ex-clerk to the Commons, and Glyn Mathias, a former electoral commissioner, claim not enough provision has been made for voters to sign the petition.
They have written to the chief executive of Powys County Council and Petition Officer, Dr Caroline Turner, and Bob Posner, chief executive of the Electoral Commission, complaining about the procedure.
The letter highlights defects in the procedure for managing this recall petition and the degree of public confusion about how the electorate are supposed to sign".
Sir Paul and Mr Mathias say they are concerned by the decision to "nominate only six locations against a legal maximum of 10".
"Brecon and Radnorshire is a highly rural constituency - the largest geographically in England and Wales - and the distance many electors would have to travel is a significant deterrence to signing.
"We note that the maximum of 10 locations were provided at the previous recall petition at Peterborough [where former Labour MP Fiona Onasanya was removed from office about serving a prison sentence for lying about a driving offence], even though it is a much smaller constituency."
The letter also expresses concerns about the times the signing stations are open, claiming they are "very inconvenient for those who work".
It adds: "We are concerned that the combination of the lack of an adequate number of signing places and the hours at which they are open may call into question the fairness of this recall petition."
Petition Officer Dr Caroline Turner said: "All arrangements for the Recall Petition are in full
accordance with legislation under the provision of the Recall of MPs Act 2015.
"Information about the allocated signing locations and opening times are available on our website."
A spokeswoman for the Electoral Commission said: "Responsibility for allocating the signing
places and times of opening for the recall petition lies with the Petition Officers for the constituency
.
"However, any electors unable to attend the signing places in person also have the option of signing by postal or proxy vote.
"They can find more information on how to apply for a postal or proxy vote on the Powys County Council website.
"The Commission has a statutory duty to report on the administration of the recall petition and we will consider the accessibility of signing places as part of that report."