| Things TODO | |
| =========== | |
| * Fix the numerous malloc+copy operations for sending data, see "Buffering | |
| Improvements" below for details | |
| * make sure the windowing code adapts better to slow situations so that it | |
| doesn't then use as much memory as today. Possibly by an app-controllable | |
| "Window mode"? | |
| * Decrease the number of mallocs. Everywhere. Will get easier once the | |
| buffering improvements have been done. | |
| * Use SO_NOSIGPIPE for Mac OS/BSD systems where MSG_NOSIGNAL doesn't | |
| exist/work | |
| * Extend the test suite to actually test lots of aspects of libssh2 | |
| * Fix all compiler warnings (some can't be done without API changes) | |
| * Expose error messages sent by the server | |
| * select() is troublesome with libssh2 when using multiple channels over | |
| the same session. See "New Transport API" below for more details. | |
| At next SONAME bump | |
| =================== | |
| * stop using #defined macros as part of the official API. The macros should | |
| either be turned into real functions or discarded from the API. | |
| * fix the parts of the API where object pointers and function pointers are | |
| mixed like libssh2_session_callback_set() | |
| * remove the following functions from the API/ABI | |
| libssh2_base64_decode() | |
| libssh2_session_flag() | |
| libssh2_channel_handle_extended_data() | |
| libssh2_channel_receive_window_adjust() | |
| libssh2_poll() | |
| libssh2_poll_channel_read() | |
| libssh2_session_startup() (libssh2_session_handshake() is the replacement) | |
| libssh2_banner_set() (libssh2_session_banner_set() is the repacement) | |
| * Rename a few function: | |
| libssh2_hostkey_hash => libssh2_session_hostkey_hash | |
| libssh2_banner_set => libssh2_session_banner_set | |
| * change 'int' to 'libssh2_socket_t' in the public API for sockets. | |
| * Use 'size_t' for string lengths in all functions. | |
| * Add a comment field to struct libssh2_knownhost. | |
| * remove the existing libssh2_knownhost_add() function and rename | |
| libssh2_knownhost_addc to become the new libssh2_knownhost_add instead | |
| * remove the existing libssh2_scp_send_ex() function and rename | |
| libssh2_scp_send64 to become the new libssh2_scp_send instead. | |
| * remove the existing libssh2_knownhost_check() functin and rename | |
| libssh2_knownhost_checkp() to become the new libssh2_knownhost_check instead | |
| Buffering Improvements | |
| ====================== | |
| transport_write | |
| - If this function gets called with a total packet size that is larger than | |
| 32K, it should create more than one SSH packet so that it keeps the largest | |
| one below 32K | |
| sftp_write | |
| - should not copy/allocate anything for the data, only create a header chunk | |
| and pass on the payload data to channel_write "pointed to" | |
| New Transport API | |
| ================= | |
| THE PROBLEM | |
| The problem in a nutshell is that when an application opens up multiple | |
| channels over a single session, those are all using the same socket. If the | |
| application is then using select() to wait for traffic (like any sensible app | |
| does) and wants to act on the data when select() tells there is something to | |
| for example read, what does an application do? | |
| With our current API, you have to loop over all the channels and read from | |
| them to see if they have data. This effectively makes blocking reads | |
| impossible. If the app has many channels in a setup like this, it even becomes | |
| slow. (The original API had the libssh2_poll_channel_read() and libssh2_poll() | |
| to somewhat overcome this hurdle, but they too have pretty much the same | |
| problems plus a few others.) | |
| Traffic in the other direction is similarly limited: the app has to try | |
| sending to all channels, even though some of them may very well not accept any | |
| data at that point. | |
| A SOLUTION | |
| I suggest we introduce two new helper functions: | |
| libssh2_transport_read() | |
| - Read "a bunch" of data from the given socket and returns information to the | |
| app about what channels that are now readable (ie they will not block when | |
| read from). The function can be called over and over and it will repeatedly | |
| return info about what channels that are readable at that moment. | |
| libssh2_transport_write() | |
| - Returns information about what channels that are writable, in the sense | |
| that they have windows set from the remote side that allows data to get | |
| sent. Writing to one of those channels will not block. Of course, the | |
| underlying socket may only accept a certain amount of data, so at the first | |
| short return, nothing more should be attempted to get sent until select() | |
| (or equivalent) has been used on the master socket again. | |
| I haven't yet figured out a sensible API for how these functions should return | |
| that info, but if we agree on the general principles I guess we can work that | |
| out. | |
| VOLUNTARY | |
| I wanted to mention that these two helper functions would not be mandatory | |
| in any way. They would just be there for those who want them, and existing | |
| programs can remain using the old functions only if they prefer to. | |
| New SFTP API | |
| ============ | |
| PURPOSE | |
| Provide API functions that explicitly tells at once that a (full) SFTP file | |
| transfer is wanted, to allow libssh2 to leverage on that knowledge to speed | |
| up things internally. It can for example do read ahead, buffer writes (merge | |
| small writes into larger chunks), better tune the SSH window and more. This | |
| sort of API is already provided for SCP transfers. | |
| API | |
| New functions: | |
| LIBSSH2_SFTP_HANDLE *libssh2_sftp_send(SFTP_SESSION *sftp, | |
| uint64_t filesize, | |
| char *remote_path, | |
| size_t remote_path_len, | |
| long mode); | |
| Tell libssh2 that a local file with a given size is about to get sent to | |
| the SFTP server. | |
| LIBSSH2_SFTP_HANDLE *libssh2_sftp_recv(); | |
| Tell libssh2 that a remote file is requested to get downloaded from the SFTP | |
| server. | |
| Only the setup of the file transfer is different from an application's point | |
| of view. Depending on direction of the transfer(s), the following already | |
| existing functions should then be used until the transfer is complete: | |
| libssh2_sftp_read() | |
| libssh2_sftp_write() | |
| HOW TO USE | |
| 1. Setup the transfer using one of the two new functions. | |
| 2. Loop through the reading or writing of data. | |
| 3. Cleanup the transfer |