Hi folks - Thanks for taking a look at Azure notebooks. Here's some background info:
What is it?
It's an offshoot of Azure ML Studio which has Jupyter support. We asked the powers that be if we could also instantiate it also as a free service - "yes" was the answer.
Who is it aimed at?
Students, faculty, casual users, folks that want to give webinars, classrooms, etc & want to skip software install headaches.
What does it support?
Currently R, Py2, Py3, F#. Python is backed by the Anaconda distro. More kernels will be added based on user feedback. Environment runs on Linux/docker.
Is it free?
Yes.
I need an account?
Not to view content (a la nbviewer). To run, create notebooks, etc. you need a Microsoft account (xbox, outlook, hotmail, ...). Sample notebook to view (click on the eyeball):
We are admittedly not UI people and are grateful to our summer intern for the current UI! Please send feedback to nbhelp@microsoft.com and we'll improve it!
[EDIT: additions:]
Can I get a bash prompt, install linux pkgs?
Yes! In Jupyter, you can click on Terminal & you are in bash (ubuntu).
Can I use pip, install.packages(), nuget, ... ?
In Python/R/F#, you can use each environment's pkg mgr to install pkgs. EG "!pip install pkg", install.packages("ggplot2"), etc. See Py examples here:
No thank you! I have been running a server on my own server at home, but this has so many awesome possibilities. I can see this as my go to for introducing Python or R in the near future.
[msft] Yes! Azure N-Series was announced recently and we hope to make it an option (HW selection in general). Whether it'll be free or not is above my pay grade. Would be nice to have a free tier at least (time/size).
Giving away access to a public Jupyter Server (open source project)
Running it in Linux (Ubuntu) and letting users have a play with it.
I for one am super happy to play with it however long this lasts.
Some info so far.
!free -h
total used free shared buff/cache available
Mem: 55G 7.5G 26G 70M 21G 44G
Swap: 99G 0B 99G
!uname -a
Linux nbserver 4.4.0-51-generic #72-Ubuntu SMP Thu Nov 24 18:29:54 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
Look! I can even install stuff I want from pip...
!pip install terminaltables
Collecting terminaltables
Downloading terminaltables-3.1.0.tar.gz
Building wheels for collected packages: terminaltables
Running setup.py bdist_wheel for terminaltables ... - \ done
Stored in directory: /home/nbuser/.cache/pip/wheels/96/0c/9a/0ec2bcad2ac1fb1d0e4695879386460cec1a947cd7413d1b17
Successfully built terminaltables
Installing collected packages: terminaltables
Successfully installed terminaltables-3.1.0
You are using pip version 8.1.2, however version 9.0.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
[msft] :). Interestingly enough, the decision to use linux/docker was a no-brainer. I know from the outside it might seem odd, but these days, the attitude (and rightly so) is: use the language/stack/OS/toolchain/etc. you need to get your job done. I think you'll see an explosion of Linux/Python/Java/Go/... internally & externally on Azure.
Data point - our team (Python), wanted to hold a "Python day @ msft" event to teach/inform internal teams on the language/ecosystem. We were hoping maybe 50..100 would show up, if lucky. A thousand and twenty one people showed up!
Another data point - the official Azure CLI was just rewritten in Python (win/macos/linux):
It does make so much sense. Makes it easier and quicker to get things out the door.
I'm a performance guy at my work responsible for making sure our Filesystem (Huge codebase of C, and bits of C++), making use of Python to quickly write scripts for Performance data analysis. Right tool for the job.
It is interesting to think about how many use cases there are in this ecosystem where that hard attention to performance is "wanted" but not needed. Could I write my scripts in a complied language and save two seconds of execution time? Yes, but I finished two more tasks in the time it would've taken to do it.
Maybe it runs on Linux because jupyterhub does not work on Windows, not even on Linux subsystem? Microsoft ported docker to Windows, please do the same with jupyterhub!
[msft] Hi dnfromufa. Actually the decision to go with linux/docker was purely because, (at least) at that time, container mgmt was better on linux. In fact Azure ML backends run Windows and you can use notebooks to build ML models, then deploy to windows backends. Thank you Python & Anaconda xplat support!
RE Jupyterhub on Windows - if enough customers want it, I'm sure it can be done. Not convinced the demand is there yet. If you have data I'd love to see it!
[msft] Thanks for digging! Honestly I understand as their team is actively trying to make things like jupyterlab work well right now. My guess is that if services like ours become wildly popular, someone (msft?) may step up and do & maintain the Windows port. I'm sure it'll be popular at Wall Street etc type shops. TBD.
I've been experimenting with Jupyter notebooks recently so I thought I'd try uploading one I made which solves a wooden puzzle. It seemed to work just fine!
I had to host the images in the notebook elsewhere rather than upload them with the notebook. It looks like that is possible but then I'd have to grapple with Azure blobs...
That's a great use of Jupyter! You've taken a concrete problem, the wooden puzzle, and written a well explained software solution to solve it. Anyone can come behind you and add a 3d graphic or analyze the space and time utilization of the algorithm.
Working with Azure Storage Blobs is pretty straightforward - you can use the cross platform azure CLI application or use the Azure Storage Explorer application (which appears to be cross-platform as well in its latest form):
I've been using the linux version for a couple months now, and it's a truck ton better than exploring through VS. I'm still waiting for it to support tabs, or any other multi-tasking interface. That's sort of a pain.
for those on windows and don't mind dropping a little $, I've been relatively happy with CloudXPlorer http://clumsyleaf.com/products/cloudxplorer for browsing blobs. Has most of the features I care about
TableXPlorer (for Azure Table Storage is decent too)
Pretty surprised at this as well. But, given you know this requirement now, is it safe to assume that if you embed remote images in your notebooks in future and upload them you'll have no issue?
(i.e. it's not stripping them out and making you manually re-embed them each time if they aren't an Azure blob?)
It's really lovely when a tool written in another tech (Python there) can be used/extended with others stack/tech, because was written as language agnostic. That should be a rule for good design in oss tooling. Not starting using language specific communication, but extensibile from day0 in design (obv default language can be bundled)
Thanks enricsosada.
I am a dev on Azure Notebooks and I spent time working with a couple of people in Microsoft Research to get this going and we are very happy with the result. The F# community has been very helpful in getting this going.
...makes anyone say "yuck" and avoiding F# for any exploratory work. And even after you understand how the pipe operator works and how useful it can be in other contexts. I mean, ugh... Even putting up with superfluous verbosity like `List.Map(myList)` instead of a `map(myList, ...)` or `myList.map(...)` because "that's how F#/OCaml does it, stfu" is ok, but this seems like purposeful obsfucation.
No wonder R and Python are the only languages popular in ML. They are the only ones leading to sane readable code by people having other stuff to keep loaded in their head than language details. I mean, yeah, syntax that doesn't matter, unless it's so f annoying and ugly that you simply can't put up with it as much as you try!
I'm... confused what you're upset about. Are you upset about the leading "Chart?"
This is very, very easy stuff for functional programming. Most of the people I know in ML, talented ML folks doing data science within the org I work for to do exciting things... they actually aren't huge fans of the way Python does things. They're huge fans of the libraries Python has to speed up ML work. F# has a real lack there (surprisingly!).
But like, saying that you like:
Chart.Line(data).WithTitle("smth").render()
as opposed to:
Chart.Line(data) |> Chart.WithTitle("smdh")
seems to me like 6 of one or half a dozen of the other. Especially since when you're reusing any given part of the pipeline, the ML way is alot more terse.
And it's not like Python is devoid of hideous examples of poor language design. For example, self parameters. All we're told is some new age mysticism about how "explicit is better than implicit", but somehow that doesn't apply when we need higher order functions and we're using private function with lambda names to try and signal to readers that, "Sorry I needed 2 statements in my list comprehension and Python talks down to me like that guy from Timecube because of it." Why is explicitness valuable there but not on tuple construction?
And it's not like Python doesn't make you arbitrarily choose between List.function and list.method with no seeming decisions to make.
Which is not to say Python is especially bad. It is to say that you may have normalized Python so much that you forget all computer languages are somewhat arbitrary.
Yeah, I'm fairly certain that guy was trolling, but to make this post actually worthwhile, there are a ton of ML libs accessible from F#, http://fsharp.org/guides/data-science/
Lists a bunch though the R interop is probably the best/most fully formed IMO.
No, avoiding unnecessary repetition (god damn DRY at the syntax level) and avoiding visual noise is not arbitrary. It's just good practical taste. I don't live in houses where doors have 2 knobs that need to be pressed at the same time to open a door. And I don't eat steak with a fork that has a knife as a handle. Both of those could work and make sense for some, but overall they are bad taste and awkward for 99% of people.
Python gets it right. Ruby gets it right. Julia gets it right. Most lisps get it right etc. (Even Java and C# get them "as right as they could" considering all their history and past choices that seriously limited what kinds of languages they could be.)
And my issue is that I like the semantics of languages like F# and OCaml and Haskell after having played a bit with them but by god, they couldn't have chosen more infuriating syntaxes and name resolution systems or module systems or tools for them... like they tried as hard as they could to piss off "the plebs in the industry" who actually care about syntax and other such little details, because, ya know, when what you're developing is not that interesting, you should at least have the pleasure of writing code that you aesthetically enjoy to read! And it's hard to convince fellow industry plebs of the usefulness of advanced type systems when the first code sample they see elicits an "ugh" or "yuck" reaction. Most of us programmers are shallow and lazy and we should be proud of this and build tools that cater to our "virtues".
Or maybe I'm in the minority by liking dense and non-repetitive notations and finding them easier to read too...
> Python gets it right. Ruby gets it right. Julia gets it right. Most lisps get it right etc.
What. I ship Clojure all day, got CL in my past. Was a full time ruby dev. I got a list of ugh and eck for all of them. I can name 30 more issues with Python. Truth be told, I think Python is a rancid language and I think people who love it are basically eating barf pancakes every day and thanking people who look down on then for it. GVR doesn't do that bs "I don't get lambdas" garbage in the company of other language designers, that's for sure.
What you consider visual noise is arbitrary. It's like what color paper you prefer to note take on. Every language has issues. Python is riddled with syntactic noise and artifacts that you've normalized. Ruby's syntax is better, but still full of quirks and surprises. Don't even get me started on Scala.
"I am used to this" and "this is objectively better unless you are some academic" is a classic example of industry insecurity. You shouldn't avoid Haskell because you are irritated with a bit of syntax, you should avoid it because you can't ship or maintain the kind of deliverables you need to write.
> Most of us programmers are shallow and lazy and we should be proud of this and build tools that cater to our "virtues".
Pride is one of the ugliest sins of our industry, I agree. Too many developers refuse to accept that there might be progress in the industry outside of what they experienced in their first 2 years in the industry.
While his vitriol was extremely hyperbolic, I get his point somewhat on obtuse syntax.
Take this example of Haskell from their docs:
thenP :: P a -> (a -> P b) -> P b
m `thenP` k = \s ->
case m s of
Ok a -> k a s
Failed e -> Failed e
It doesn't matter how good you are in [insert almost any common language], you'll really struggle to understand that code.
It's my same objection to things like Coffeescript, if you think you are so much more productive not typing semantic tokens like ()s or {}s, then you need to lay off the coffee and get some sleep as you're clearly dillusional.
> It doesn't matter how good you are in [insert almost any common language], you'll really struggle to understand that code.
To the extent that true, it's because of the weird historic moment between the mid 1990s and now where essentially every industrially popular general purpose language is from the C branch of the Algol family.
It's not really the whitespace (which is clear in semantics), and other than the lambda slash and the infix ticks, the whole thing is pretty clear from a Lisp background. (Well, I'm familiar enough with Haskell now that I'm not filtering it through some other language, but the similarity to Lisp is what helped it start to click early on for me, even though it's an ML descendant and not a Lisp descendant.)
We're actually starting to see more gains for non-Algol descended general purpose languages, so maybe the syntax shyness of the last generation or so of programmers will soon be a thing of the past, because knowing multiple languages won't so invariably be knowing multiple members of the same syntax family.
If you removed the middle paragraph, I'd wholeheartedly agree everything you said. Unfortunately I don't feel my prior Lisp experience helped me as much when I was starting off in Haskell as it apparently did you.
And it looks slightly clearer when laid out correctly
thenP :: P a -> (a -> P b) -> P b
m `thenP` k = \s ->
case m s of
Ok a -> k a s
Failed e -> Failed e
But this is not a fair test. There needs to be a control. Implement the same functionality in Python and then we'll talk about which language is clearer.
I copy-pasted directly from the doc page that layout, guess I should've proofread it to make sure alignment was right. Thanks!
As to the comment re: control / python, I'm sorry but it's so obvious to me the syntax would be more legible to most developers (since "most" use Algol descendent syntax languages) even if done somewhat poorly, I don't feel like taking the time. I welcome you to prove me wrong, I'll gladly stand corrected if so!
Yes, it looks clear to me too. But you and I are Haskell programmers. That said, outside maybe OCaml, no other language I've written code in (which is about 26ish at this point) prepared me mentally to be able to grok the syntax.
> You shouldn't avoid Haskell because you are irritated with a bit of syntax, you should avoid it because you can't ship or maintain the kind of deliverables you need to write.
I can ship and maintain deliverables I need to write in Haskell more effectively than other languages.
Sure, and that's great. Not everyone had a domain where Haskell is appropriate or access to a group of people who can help with maintenence in Haskell.
And some domains are ill-suited to Haskell due to constraints on the VM.
> you should avoid it because you can't ship or maintain the kind of deliverables you need to write.
To clarify, you were saying "you should avoid Haskell in the case you can't ship or maintain the kind of deliverables you need to write" and not "you should avoid Haskell because Haskell won't let you write or maintain the kind of deliverables you need to write"...
This is a really interesting comment for me to read. I chose Python as a language to work in professionally in part because of its lovely syntax and the way it supports removing duplication. Now I've moved to Haskell and I'm even happier with the syntax and the support for removing duplication.
> No wonder R and Python are the only languages popular in ML
R is a curious counter-example, given that the '%>%'[1] operator in dplyr is nearly identical to the '|>' operator in F#. Given dplyr's ubiquity, it would seem that people coming from R would have no issue with F#.
I am a huge fan of Jupyter notebooks, the idea of being able to run my code easily alongside natural prose is crucial when telling the story of data.
But, Jupyter faces most of it's challenges in the word-editor part of their product rather than in the code part.
I'd love to see a partnership between them and the Eve programming language (http://witheve.com/) who have absolutely mastered the IDE interface in early renditions of their product.
I know a lot of people are against the concept of Eve from a purist perspective of code not needing to be humanized. But, one of the common goals in data is to communicate insights and solutions through visually crafted and accessible stories. I think there is still a long way we can come in that.
I listened to a podcast about jupyter [1] some time ago, and remebered them talking about the separation between the kernel and the presentation in jupyter, and that in theory, it should be relatively easy to write your own frontend, that just sends to correct messages to the zmq :-)
Afaik, hydrogen plugin for atom does this [2]
And there is another plugin that just wraps the notebook to live inside of atom [3]
I absolutely LOVE Jupyter but I personally have found something better. I also love that Microsoft is behind making this happen with Jupyter.
RStudio's new feature to R is Notebooks and it is available in RStudio version 1.0+. It takes what is great about Jupyter Notebooks and adds easier version control and much easier to batch process your reports. Which are both huge wins for me.
As an authoring format, R Markdown bears many similarities to traditional notebooks like Jupyter and Beaker. However, code in notebooks is typically executed interactively, one cell at a time, whereas code in R Markdown documents is typically executed in batch.
R Notebooks bring the interactive model of execution to your R Markdown documents, giving you the capability to work quickly and iteratively in a notebook interface without leaving behind the plain-text tools and production-quality output you’ve come to rely on from R Markdown."
Right, this notebook or worksheet style interface isn't meant to be a text editor tho', or to develop software it's meant to interactively help you write a paper or a report. We had this in MathCAD in the '90s, where it used MS Word as it's "editor" https://en.wikipedia.org/wiki/Mathcad
And btw, it's worth mentioning that in JupyterLab, we just merged a PR that will make embedding the Monaco editor (the editing component of VS code) much easier: https://github.com/jupyterlab/jupyterlab/pull/1140.
We all want stronger editing capabilities, but it doesn't make sense for the Jupyter team to get into the business of writing text editors (plenty of better folks doing a great job on that already). So we're just trying to make it easier to integrate other text editors into the everyday workflow.
[msft] Well, this is an honor - the creator of Jupyter commenting on this thread & not just any comment... but about embedding the Visual Studio Code editor in Jupyter Lab :).
Thanks Fernando, Brian, Min & team for everything you've done with Jupyter. Looking fwd to using Jlab soon!
+1 to the team that's been doing a phenomenal job on the project... These days I'm just an email answering machine :)
But yes, JLab is shaping up quite nicely, opening up a lot of interesting possibilities. For advanced users/early adopters I think it's time to start playing with it (and filing issues for anything that's broken/sub-optimal, we really want to provide a great user experience with it once we hit 1.0).
Given that Jupyter works on a server / browser model, it should be possible to customize the browser without bothering the server. For instance, I imagine a plug-in that spiffs up the editing blocks within one of the major browsers. That way, you preserve compatibility across all browsers, but provide an option for those who want their preferred editing features.
Admittedly, I don't use a code editor, perhaps because I'm just too old, and got used to living without one. I remember when being able to edit a program in full screen mode was a big deal. But I certainly wouldn't turn down better editing for Jupyter.
I agree about the editor part, it's quite bad compared to anything people actually use when developing software.
...but perhaps software developers aren't the target audience in the first place. I tried to use Jupyter a few times, first when it was still only IPython, and it never seemed to fit in my code-execute-fix workflow that you have when developing scripts. In particular having to always reset the kernel to reexecute everything from scratch drove me crazy.
I'd argue that it's not really meant for developing software. It's meant for data analysis. When I do my analysis, I am not really concerned with developing the script/algorithm. I am mainly concerned with manipulating or visualizing the data. In my opinion, anything that takes more than an hour to process in the notebook should probably be done outside the notebook as an independent script.
The only thing that's really missing for me is a more persistent data store in between kernel restarts. If it took more than 5 minutes to run something to transform or process my data, I don't want to have to redo it when I restart the kernel. I think there are a couple of plugins that handle this for you, but it would really be nice if it was implemented natively. The solution right now just seems to be producing a bunch of intermediate files that you reload when you restart the kernel.
The thing is, anyone of us that had the pleasure to work in commercial Common Lisp, Smalltalk or Oberon environments already experimented live a Jupyter-like experience.
Or had access to papers and videos showing how the development environments at Xerox(Lisp, Mesa/Cedar, Smalltalk) and Genera (Lisp) worked.
Hence why I am not found of having a graphics workstation full of xterms.
We need to make these workflows mainstream, not something that our descendants are reading about in paper and videos.
I would argue that the problem is that it IS targeted toward developers. In the sense of those who are developing new methods and making proof of concepts.
Admittedly, most of my experience is with ipython but it is perfect for that. If there is a new algorithm/method I want to explore or I am figuring out an interface/structure, it is great. And I tend to not have to reset the kernel too often.
In that sense, jupyter was a great idea as you can now integrate documentation with code in a nice format.
Unfortunately, it also led to a strong focus on treating them as containers for the purpose of deploying code (more traditional software development).
Do you know of a better tool for such a flow, combined with the ability to have markdown+latex docs intermingled with the code? 'cause this is my workflow, but jupyter's code editor and kernell restart drives me crazy and anything else will have me keep the notes/docs separate from the code...
And please don't suggest Mathematica :) I absolutely love it's ux/i, but nobody uses it in ML/AI...
You can use whatever editor you want, but I quite like rstudio (I'm generally not a huge R person, so an environment with more help is useful, whereas with python I'd prefer just my own setup).
Edit - Importantly though, you actually don't need to use R, you can use python. I'm not sure how well that works with caching, as I've never tried it, but it's probably worth a go.
hmm... thanks for making me take another look around. I'm trying rstudio+python and rodeo tonight to see if I like them better.
(Right now I use jupyter for some things, ipython gui for others, and pycharm for "real coding" tasks. Tried Spyder, but something about it makes it neither a good IDE nor a good notes/documentation system... though I can understand its appeal for Matlab folks).
[msft] You are quite correct that the REPL/cell paradigm has its pluses & minuses. That's why the follow on (Jupyter Lab), is addressing many of these issues. Check out:
the UI is really bad.
Idea is great from Microsoft but the execution leaves alot to be desired. I hope MS fixes this because I kinda like this service and can see myself using it.
Thanks for feedback. We hope to keep iterating on the UI in the coming year. We'd love to improve it based on your suggestions (we're admittedly weak on that front). nbhelp@microsoft.com.
Hi hatmatrix - Currently it's primarily a hosted jupyter service - you can view with no login, or create/run if you login. It does not modify Jupyter itself in any significant way. We hope to add various features in the coming year. In its current form it can be used as a scratchpad, for classes, webinars, conferences, etc. where you have say 100+ ppl that quickly need to follow a tutorial without spending time on installation (it has Anaconda sitting underneath Python).
Just a note, if you have F# you can install Mbrace. Which means you can actually split workflows arbitrarily from inside the notebook. That's really, really powerful.
There are a number of projects for transpiling X language into JavaScript. Most aren't super robust, but generally good enough for basic purposes. I feel once WebAssembly gets more steam, this will get even more robust.
Not claiming this is a good solution, but definitely would be one possible route.
One of my physics major friends uses a Chromebook (and a shitty Windows laptop he prefers not to deal with). I set up a DigitalOcean droplet for him to run Jupyter. One of the big problems is that it's not safe for him to share Jupyter notebooks by URL, since anyone to whom he discloses the web UI password gets remote code execution (exec and friends work).
A professionally maintained, sufficiently sandboxed Jupyter environment could be awesome for people who want to work on and share Jupyter notebooks, but not be responsible for servers.
Very interesting. I just put some R code into production on Azure via MS R Server (DeployR), and it was not a particularly fun experience. I really want to see them do a hosted R service where we can just take these Jupyter Notebooks and expose it as a web service on a PAYG basis.
We looked at that, in fact it was my first recommendation, but the way it forces you to set things up is less than ideal. We may give it another go, but it was not conducive to just taking the data scientist's R code and exposing it as a web endpoint - it requires a complete restructuring of the code.
OK this is freaking awesome. I've been working on some Machine Learning courses and they use Jupyter. I have the issue where whenever I switch computers I have to sync files and all that jazz. I'm so excited about this tool I can't even contain myself.
The signup you need is a Microsoft account. This is the same account you use for Xbox Live, hotmail/outlook, etc. We have found most people already have these accounts. It also works with organizational accounts so a good number of college email addresses work if your university has taken the time to set it up. Plenty of university professors have been trying out our platform with their classes. They have had little issues with the login barrier.
The user that posted before me is correct that this is a calculated decision. Some of it is for security considerations but also just so we can return a user to their previous content and not have to field questions on 'where did my files go' when they weren't logged in before and so they didn't persist.
Having taught after school programs for kids, it definitely is a barrier to entry. I'd not say it's a show stopper, but it's definitely a hassle. Even more relevant when the folks needing to sign up, "can't" easily (due to COPPA).
[msft] Understood. Note that you can view notebooks w/o login. If you want a truly no login experience, try the awesome https://tmpnb.org service. But note that these are temporary (tmp-nb...) and will go away. The login we require is to show you your notebooks next time you're back. If your school has office365 the existing creds should just work.
I fully understand you can view, but "running" code is the entire point of an after-school computer programming class. As these are after school classes (but done in collaboration with the school), some resources the school has aren't available.
Also, how many middle school and high schools (especially title 1 schools) do you know have office365 for the teachers, let alone for each and every student?
Please don't regard this reply as being critical of the offering, it's certainly not and I applaud what you're doing, I'm more responding to the person who couldn't understand why login was a barrier.
[msft] Fair points jsjohnst. I don't have a good solution for you. Perhaps the teacher can setup one Microsoft account (eg school.outlook.com) and multiple ppl can use it? (but that comes w issues too sadly).
This is amazing, I had set up a jupyter hub using some docker hacks to allow my students to access their own personal instance(running inside a new container) using github login. This might be a good low-cost alternative. Thanks Azure!
Ok, let's use OneDrive to load data. Where is that option?
Nvm, let's use Dropbox instead. Does not work "Something went wrong. Please refresh the page."
Maybe Azure storage? Trying to sign-up, filled sign-up form, verified account via SMS, entered CC data. Nope! "Cannot proceed with signup due to issue with your account. Contact support". (tried multiple times)
Contacting support.. creating incident, describing the issue. When trying to attach screenshot I get "The file upload service server is not available at this time. Wait a few minutes and then try it again."
[msft] RE Dropbox - Very sorry about that :(. We discovered an issue earlier and are feverishly looking at fixing it. Will remove the UI until it is fixed. Meanwhile you can !curl in data among other approaches.
RE Storage/CC sign up. I'll locate someone in that group & reply back.
This looks really nice. I've always hated how buggy the free hosted Jupyter notebook services are, and setting up a remote server yourself is definitely not a one-click experience. Surprised to see how unrestricted this is (for now anyway). They'll probably see some abuse of the resources soon and add more restrictions. The "etc" from the restrictions section of the faq is pretty broad.
"Usage should be limited to learning, research, general computing, etc."
Thanks sixhobbits.
Our goal isn't to restrict you if it isn't required. We do need to maintain the right to stop people from abusing this and make sure there is room on the service for multiple users. Aside from that you are basically free to party on :)
Pretty interesting as a tool to learn Python or other things. I'm intrigued.
I'm doing something similar with exploit development learning, but with a javascript based terminal and linux containers, and a markdown writeup (https://exploit.courses for anyone interested). But the close interaction of code and text in Jupyter is much more advanced, and useful :-)
If you don't access your account for 60 days we reserve the right to remove your data (currently Notebooks but eventually notebooks, csv files, etc). This is to avoid us wasting storage on users that aren't actually using the service.
As we are still new, we haven't started deleting files yet, but we wanted to be sure to have a policy for handling this upfront and not decide when there was a problem later and then start destroying content without any sort of notice.
[msft] Proper data support is coming. I believe it's meant as: if you upload a ton of data & seem to have abandoned it, it'll eventually go away. Admittedly vague, will be made more crisp in the faq. Thanks!
I have not visited any Microsoft property for some time now and I have this strange feeling about the UX. Sadly I can not say if it is my deeply rooted aversion against everything Microsoft or if the UX is really odd.
Hi dandare. I am a dev on Azure Notebooks at Microsoft and can tell you that our team also delivers Python Tools for Visual Studio, Azure SDK for Python, R Tools for Visual Studio, and previously Node.js Tools for Visual Studio.
The UX was not done in any way to feel like something else. We tried to design something that worked well for Jupyter users as well as looked nice and was usable.
That being said, our teams strengths are not in UX by far. We are starting to work with designers to improve the "dev-UI"" we currently have.
If you have any more feedback you would like to share we would love to hear it. There is a public tracker at https://github.com/microsoft/azurenotebooks and an email that goes to the team at nbhelp@microsoft.com
As chupapuma said, we aren't UI folks and all feedback is welcome. We hope to update it gradually in the coming year :). The original UI was done by one of our summer interns. [EDIT] didn't want to make it sound like I'm blaming our intern - he was awesome, especially if you'd seen the previous rev!
I think your impressions of Microsoft are out-of-date. We've been successfully using Azure resources through "Azure for Research" grants. My world, is the world of Linux VMs, Docker, and OSS and I find Azure to be a good fit for our research needs. (I never touch any Windows OS, if that's what you are worried about.)
I was in a similar situation a short time ago. Although I found it required a lot of screen space to work well, it could be a little slow, and it took some getting used to, it really grew on me. The hierarchical drill-down was really nice once you had a lot of different types of resources. I like AWS' interface, but sometimes I miss the feel of that navigation...
It took me 55 seconds to load the page (42 of which went into downloading a 6 MB large GIF). 40+ requests. And it doesn't seem like a slow Internet problem at all.
Seems like they did something to the notebook interface to make it more responsive on tablets. I usually cannot manipulate Jupyter notebooks from an iPad, but I can here. Is this just a MS modification?
Jupiter is a great project for data scientists. Here is a one sentence explanation from jupyter.org:
"The Jupyter Notebook is a web application that allows you to create and share documents that contain live code, equations, visualizations and explanatory text."
Sorry to sound cynical but haven't they taken something that's existed for a long time and spent money on marketing it to add / repair value to their brand? It feels a little obvious to me to be honest?
Actually, this is somewhat surprising. It's taking what many, MANY data scientists and analysts are actually using day-to-day and making it point-and-click to start in the cloud.
You can set these things up in AWS and GCE, but it requires a lot more work. Here is my top google hit for setting things up on AWS, from a relatively recent and maintained repository: https://gist.github.com/iamatypeofwalrus/5183133
Many data scientists just run their notebooks locally. Few have the luxury of Data Science Engineers on hand to help them through the complex world of provisioning and maintaining servers, even in this canned AWS world where spark clusters unfold themselves in EMR.
Azure is absolutely full of "here is a canned solution for _insert common engineer task with open source focus here_" buttons and it's really quite great as a result.
Also: Azure is doing a very good job of making it difficult to have a default-insecure instance. You don't make choices about encryption-at-rest, for example. That's forced on you. There is a lot more tooling around sharing accounts and giving users minimal ACLs rather than just doing the simple thing in AWS or GCE and giving out power user accounts.
It's simple stuff, of course, but this simple stuff is the simple stuff AWS has elected not to do, so...
After that we loved the project (and the team) so much that we donated a $100,000 for furthering development. They used the funds to hire a dev & help improve the Jupyter for all platforms.
Since then we've put Jupyter in Azure ML studio which combines a drag/drop ML canvas with that of a notebook. You can use it to analyze your data / build models /deploy to Azure.
As an offshoot of the above project, we wanted to give something back to the community, so we figured we'd instantiate it as a separate notebook-centric offering for dabblers, students, faculty, demoers, webinar/seminar speakers, etc.
And by "we", I mean a small group in Microsoft who cooked up the idea - if it turns out to be useful, we'll keep it going. If it's not, we'll shut it off. So far thought it looks like it's hit a chord - thousands of notebooks have been created just in the past month. Various schools are using it to teach cs101, eng101 courses, people are using it to demo their products/services (an executable doc page vs a static one), ...
Beyond this initial offering, we hope to add a suite of features both around and inside Jupyter. For example, we hope to be working w the core team to add debugging/profiling support soon. We also hope that the VSCode editor will be integrated for the JupyterLab environment. We'd love to hear your ideas!
I suppose it's publicly hosted which might be rare or totally new. TBH the most frustrating thing is that unlike github's notebook hosting you need to login just to view the content.
And for some reason the back button doesn't work once you encounter the login page. That's enough to make me want to stay away.
Thanks for the feedback wyldfire. I am a dev on Azure Notebooks and wanted to mention we do allow viewing the content without login. However, to be in an executable context (jupyter) we require login. Our samples, in order to avoid extra clicks, take you straight into an executable environment.
Thanks for being so responsive, Chris. I think that the 'view notebook' [eye icon] is too out-of-the-way to notice.
IMO a design that might be less surprising would be for clicking on the name of the notebook to take you to the read-only view of the notebook. From there you could click on something that would promote it to an executable/editable view and login if necessary.
But the broken back button on login page is a real bummer.
EDIT either it was my imagination to begin with or it's already fixed because I tried it again and the back button appears to work correctly.
Isn't that the point of open source? That anyone can use it for their own ends, with varying degrees of contribution back into the core (based on the license)?
A Notebook here is a polyglot, browser based, REPL on steroids.
Smalltalk is somewhat of a primitive browser and very much a REPL on steroids. Instead of being polyglot from the standpoint of supporting multiple languages, Smalltalk was designed to be usable by people from grade school students up through researchers. (It largely succeeded in that goal, but failed in that it didn't attain enough popularity with mainstream engineers/programmers.)
Some of the educational uses of Squeak Smalltalk are very much in the same vein.
[msft] Wakari is a more complete product from our partners at Continuum.io (we use their Anaconda distro for Azure Notebooks). We have various improvements planned & hope to reach parity soon!
Thanks for the feedback cbHXBY1D. We discovered this having issues just a bit back. We have some devs looking at it now and hopefully can have it up and running soon.
I actually thought this was some kind of laptop for awhile, even after reading much of the front page. I googled Jupyter Notebook after a few context clues finally tipped me off. I just thought I was being dense, though.
What is it?
It's an offshoot of Azure ML Studio which has Jupyter support. We asked the powers that be if we could also instantiate it also as a free service - "yes" was the answer.
Who is it aimed at?
Students, faculty, casual users, folks that want to give webinars, classrooms, etc & want to skip software install headaches.
What does it support?
Currently R, Py2, Py3, F#. Python is backed by the Anaconda distro. More kernels will be added based on user feedback. Environment runs on Linux/docker.
Is it free?
Yes.
I need an account?
Not to view content (a la nbviewer). To run, create notebooks, etc. you need a Microsoft account (xbox, outlook, hotmail, ...). Sample notebook to view (click on the eyeball):
https://notebooks.azure.com/library/LIGOOpenScienceCenter
I don't like the UI!
We are admittedly not UI people and are grateful to our summer intern for the current UI! Please send feedback to nbhelp@microsoft.com and we'll improve it!
[EDIT: additions:]
Can I get a bash prompt, install linux pkgs?
Yes! In Jupyter, you can click on Terminal & you are in bash (ubuntu).
Can I use pip, install.packages(), nuget, ... ?
In Python/R/F#, you can use each environment's pkg mgr to install pkgs. EG "!pip install pkg", install.packages("ggplot2"), etc. See Py examples here:
https://notebooks.azure.com/library/Intro_To_CNTK
Is my environments saved?
Currently, your notebooks are saved based on your login. Proper data, load github repo, etc. support is coming.
For a bit more info, please view the faq:
https://notebooks.azure.com/faq
Thanks! [msft]