We've updated our Privacy Policy to make it clearer how we use your personal data.

We use cookies to provide you with a better experience. You can read our Cookie Policy here.

Advertisement

Handwriting Converted to Computer Code by New Tool

A person writing on paper.
Credit: StartupStockPhotos/ Pixabay

Want a FREE PDF version of This News Story?

Complete the form below and we will email you a PDF version of "Handwriting Converted to Computer Code by New Tool"

Technology Networks Ltd. needs the contact information you provide to us to contact you about our products and services. You may unsubscribe from these communications at any time. For information on how to unsubscribe, as well as our privacy practices and commitment to protecting your privacy, check out our Privacy Policy

Read time:
 

Cornell University researchers have created an interface that allows users to handwrite and sketch within computer code – a challenge to conventional coding, which typically relies on typing.


The pen-based interface, called Notate, lets users of computational, digital notebooks open drawing canvases and handwrite diagrams within lines of traditional, digitized computer code.


Powered by a deep learning model, the interface bridges handwritten and textual programming contexts: notation in the handwritten diagram can reference textual code and vice versa. For instance, Notate recognizes handwritten programming symbols, like “n”, and then links them up to their typewritten equivalents.


“A system like this would be great for data science, specifically with sketching plots and charts that then inter-operate with textual code,” said Ian Arawjo, lead author of the paper and doctoral student in the field of information science. “Our work shows that the current infrastructure of programming is actually holding us back. People are ready for this type of feature, but developers of interfaces for typing code need to take note of this and support images and graphical interfaces inside code.”


Arawjo also said the work demonstrates a new path forward by introducing artificial intelligence-powered, pen-based coding at a time when drawing tablets are becoming more widely used.


“Tools like Notate are important because they open us up to new ways to think about what programming is, and how different tools and representational practices can change that perspective,” said Tapan Parikh, associate professor of information science and paper co-author.


Reference: Arawjo I, DeArmas A, Roberts M, Basu S, Parikh T. Notational programming for notebook environments: A case study with quantum circuits. Proceedings of the 35th Annual ACM Symposium on User Interface Software and Technology. UIST, October 2022. doi: 10.1145/3526113.3545619


This article has been republished from the following materials. Note: material may have been edited for length and content. For further information, please contact the cited source.

Advertisement