NetworkedImports

Note: This is just an idea atm, let me know on IRC if its too much crack or theres something you don't like

Networked Imports would be a nice thing to add to a language I think, this is what I was thinking for in python:

a networked import would be an import where you specify the server that its at and the interpreter would automatically fetch a specific file on that server to import it, a networked import would be specified with an @ at the end of an import statement followed by the url to a site, for example "import tfplib.cod2p@tfkyle.dyndns.org".

when import encounters a form like that it would first normalize the site url in something like the following:

• if there is no scheme (http://, ftp://, etc.) then add a default of http://

• if no path on the remote server was specified use a default name, "pypackages.list" perhaps, not sure

then import will fetch the url gotten from normalizing the site url as above, perhaps keeping a cache and on future requests preforming an HTTP HEAD request first checking the last-modified date. parse it, resolve the module uri from the parsed information and fetch it (again, caching if possible), then importing it normally (if thats possible).

the pypackages.list (or whatever) file will contain one entry per line, with each line being tab separated, currently the 2 fields I was thinking of were:

1. module/package name
2. uri to module file or package dir

(Note: I'm not sure if this would work with packages, it probably wouldn't with plain old http unless you specified all to include all modules in init.py) this can be extended easily if its needed though

an example would be the following (replace 4 spaces with a tab):

tfplib.cod2p    http://tfkyle.dyndns.org/svn/tfplib/trunk/tfplib/cod2p.py
tfplib.lamuphia    http://tfkyle.dyndns.org/svn/tfplib/trunk/tfplib/lamuphia.py
...

Problems

• r00723r0 mentioned on IRC the problem of security:

  • <r00723r0> TFKyle, WAY too unsafe
    <TFKyle> r00723r0: what if it prompted the user whenever a new module was downloaded?
    <r00723r0> TFKyle, i'm scared of the module that someone downloaded (voluntarily) is unsafe
    <r00723r0> anyone can put import os;os.system("rm -rf /") in a module
    <TFKyle> r00723r0: that risk currently exists, people can download bad modules right now
    <r00723r0> TFKyle, yes, but they can see the source
    <TFKyle> r00723r0: you can see the source this way too
    <r00723r0> TFKyle, who's going to read it
    <TFKyle> r00723r0: perhaps introduce signed modules and some sort of trust if you're that worried about it? :)
    <r00723r0> that's authentication not fucktard-proof
    <r00723r0> but authentication would be nice
    <r00723r0> you'd need to set up a protocol
    <TFKyle> its not really authentication, but meh
    <r00723r0> the user should be required to download from https or ftps
    <TFKyle> why?
    <r00723r0> that's safe?
    <TFKyle> depends on your definition of "safe"
    <r00723r0> authenticated
    <TFKyle> wtf does auth have to do with this?
    <r00723r0> so people will know the module is from the source people think it's from
    <TFKyle> Self-Signed certs are kind of dangerous if someone spoofs DNS or something
    <TFKyle> r00723r0: oh, and people read the code to everything they download and install without running stuff downloaded?
    <r00723r0> well
    <TFKyle> dunno if I want to make it idiot proof, as that would involve a lot more work
    <r00723r0> i guess not
    <r00723r0> it should be the same as downloading a module manually
    <TFKyle> though it should have some sort of security probably

• marienz says versioning is needed:

  • <marienz> TFKyle: needs at least some sort of versioning to be worh using imho
    <TFKyle> marienz: care to expand on what kind of versioning it should have?
    <marienz> TFKyle: I don't know, but with your current scheme the code doing the "remote import" and the code on the remote end need to be kept in sync. Zero api changes, or stuff breaks.
    <marienz> TFKyle: you can't ask people "What version of my dependencies were you using when it broke" this way.
    <TFKyle> true, hmm
    <TFKyle> this is probably specific to my situation but you could create separate packagelist files for each version with links to the specific "tag" in svn (tfplib.whatever http://tfkyle.dyndns.org/svn/tfplib/tags/0.1/whatever.py) and then specify packages list file in the module thats importing it, probably not the best idea