remote_api_shell.py with localhost dev_appserver

Question

How can one connect using remote_shell_api.py to a development instance of Google App Engine started with dev_appserver.py (1.9.26+)?

From the command line I get:

$ remote_api_shell.py -s localhost:8080 demo

...

urllib2.HTTPError: HTTP Error 401: Unauthorized Too many auth attempts.

When run from a script with something like this:

from google.appengine.ext.remote_api import remote_api_stub
from google.appengine.tools import appengine_rpc

def fake_auth():
        return ('pw', 'pass')
remote_api_stub.ConfigureRemoteApi(
        None, path, fake_auth, servername=server,
        save_cookies=True, secure=False,
        rpc_server_factory=appengine_rpc.HttpRpcServer
)

One gets:

google.appengine.tools.appengine_rpc.ClientLoginError: HTTP Error 403: Forbidden

I have traced back through the AppEngine code but it is not apparent whether (or how) one is to fake authentication to the local server.

When I try OAuth in the code, e.g.

remote_api_stub.ConfigureRemoteApiForOAuth(server, path, secure=True)

It throws an HTTPS error.

When I set secure to False one gets urllib2.HTTPError: HTTP Error 401: Unauthorized Too many auth attempts.

It looks like there should be a way to authenticate to a development appserver, as there is code to that effect at google.appengine.tools.appengine_rpc.py:347 but I have not yet gleaned how to employ it.

Has anyone worked this out yet?

As reported: code.google.com/p/googleappengine Issue 12465

Thanks @JoshTriiJohnston – I saw that post, but the problem is fundamentally different (i.e. connection vs. authentication), and the answer is from 2013, long before the OAuth patch. Appreciate you looking into it though. Cheers — Brian M. Hunt
– Brian M. Hunt, Commented Oct 21, 2015 at 18:10
It struck me because of you using localhost vs the app server maybe starting with a --host option. Also you should be passing an app_id when using ConfigureRemoteApi. It would be in the format: s~<app_id>. — Josh J
– Josh J, Commented Oct 21, 2015 at 18:55
Thanks @JoshTriiJohnston – thanks to your comments I figured it out -- answer below, soon. :) — Brian M. Hunt
– Brian M. Hunt, Commented Oct 21, 2015 at 21:45

Brian M. Hunt · Accepted Answer · 2015-10-22 02:20:41Z

5

The answer was to point the remote api at the API server. When one starts the appserver, it'll print something like this:

api_server.py:205] Starting API server at: http://localhost:58262
dispatcher.py:197] Starting module "default" running at: http://localhost:8080
admin_server.py:118] Starting admin server at: http://localhost:8081

Then one may connect with e.g.

    remote_api_stub.ConfigureRemoteApi(
        None, path, fake_auth, servername=server,
        save_cookies=True, secure=False,
        rpc_server_factory=appengine_rpc.HttpRpcServer
    )

where server is localhost:58262.

If the first argument is not None then it appears it must be dev~APPID (for ones own APPID)

— or —

A simpler version:

remote_api_stub.ConfigureRemoteApiForOAuth(
        server.encode('ascii'), path, secure=not local
    )

where server is the same as above, path is /_ah/remote and local is set to True when using the dev server.

If server is not ASCII encoded then hard to debug things will happen. 💀

edited Oct 22, 2015 at 2:20

answered Oct 21, 2015 at 21:49

Brian M. Hunt

84.4k76 gold badges235 silver badges349 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Josh J Over a year ago

Awesome! You use s~ when pointing to a deployed app so I thought it may be the same locally.

Brian M. Hunt Over a year ago

Thanks @JoshTriiJohnston, really appreciate your comments. Incidentally, if I recall correctly s~ is the prefix when the deployed app uses the high replication database, otherwise (i.e. it's a legacy app with master/slave) it is just the APPID.

Jay Over a year ago

I have been struggling with this too since the last SDK update. I want to just remote shell into the local dev app server. remote_api_shell -s localhost:port When I use the web server port, I get 401. Using the api server port gets around the authentication. However, If I do something, for example with an ndb transaction, I get an error from context.py. It seems like there is no context.

gae123 · Accepted Answer · 2015-12-02 08:22:11Z

4

Here is how I worked around this problem. I take advantage of a hook that the remote_api has, not sure if it is documented outside the source code, but is there.

Add the following lines in your appengine_config.py at the root directory of your app's files.

if (os.environ.get('SERVER_SOFTWARE') and
    os.environ['SERVER_SOFTWARE'].startswith('Development')):
    remoteapi_CUSTOM_ENVIRONMENT_AUTHENTICATION = (
        'SERVER_SOFTWARE', [os.environ['SERVER_SOFTWARE']])

The effect will be that your app will consider all remote API requests on the development server as coming from an admin and let them through. This should be good enough for the development environment.

answered Dec 2, 2015 at 8:22

gae123

9,6473 gold badges37 silver badges40 bronze badges

Comments

nate_weldon · Accepted Answer · 2016-09-18 23:53:02Z

0

gae123 answer work for me but I had customer authentication all ready so i used the following update

if (os.environ.get('SERVER_SOFTWARE') and
    os.environ['SERVER_SOFTWARE'].startswith('Development')):
remoteapi_CUSTOM_ENVIRONMENT_AUTHENTICATION = ('SERVER_SOFTWARE', [os.environ['SERVER_SOFTWARE']])
else:
    remoteapi_CUSTOM_ENVIRONMENT_AUTHENTICATION = ('HTTP_X_APPENGINE_INBOUND_APPID', ['appID'])

answered Sep 18, 2016 at 23:53

nate_weldon

2,3491 gold badge26 silver badges34 bronze badges

Collectives™ on Stack Overflow

remote_api_shell.py with localhost dev_appserver

3 Answers 3

— or —

3 Comments

Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

— or —

3 Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related