{"id":539,"date":"2010-03-22T10:02:47","date_gmt":"2010-03-22T17:02:47","guid":{"rendered":"https:\/\/simpleprogrammer.com\/?p=539"},"modified":"2018-02-23T16:21:13","modified_gmt":"2018-02-23T21:21:13","slug":"what-to-automate","status":"publish","type":"post","link":"https:\/\/simpleprogrammer.com\/what-to-automate\/","title":{"rendered":"What to Automate – Developer Tools"},"content":{"rendered":"

\"automate\"<\/a>Okay, so you’re convinced you need a developer tools team<\/a>, or at least that development tools are important to build<\/a>.<\/p>\n

Now, what kind of tools do you build?<\/p>\n

There is no answer that works\u00a0universally, otherwise you wouldn’t really need to build anything. \u00a0The key is finding what things your team is doing frequently and automating those things, or finding the things the team wants to do, but is unable to do.<\/p>\n

One thing we can do is look at some common areas where custom tools are likely to be needed.<\/p>\n

Logging<\/h2>\n

In many places I have worked and many software development shops I have visited, there always were plenty of logs. \u00a0The problem with logs is they tend to be large, hidden on a server somewhere, and hard to read.<\/p>\n

One area of\u00a0opportunity\u00a0is a tool to parse and extract important information out of logs and put it in a format that is more\u00a0digestible\u00a0to someone troubleshooting a problem or doing analytics. \u00a0One particular area of interest here is log messages dealing with uncaught exceptions. \u00a0It is always good to automate a way of reporting on those types of problems.<\/p>\n

Applications also often lack the log information for certain important parts of the application or processing that you only find you need once you are running in production. \u00a0Tools to instrument code and insert logging into your application while it is deployed are very useful and can be a life saver in understanding that error that only happens in production at 3:00 AM.<\/p>\n

Most web servers have logs of HTTP requests that come in from users. \u00a0This information can be very useful in determining how users are actually using your application. \u00a0Combining this information with session ids and timestamp data from your database, can provide you with exact click flows for users and answer many questions like “how the heck did this data get created?”, or “how are they creating this problem?” \u00a0Tools to aggregate this data and create representations of the user’s actions can be invaluable.<\/p>\n

Developer Build and Deploy<\/h2>\n

Developers build and deploy the application many times a day on average. \u00a0This is one of the most commonly repeated activities. \u00a0Yet, I find that in many organizations this is a complex process or takes a very long time.<\/p>\n

Consider writing tools to dynamically load parts of your application so that only that part needs to be built. \u00a0Good\u00a0candidates\u00a0here are sections of code that change frequently and are isolated.<\/p>\n

Deploying to the web server should be a one button or one command operation. \u00a0This is often a very “low hanging fruit” that can provide large benefits by saving time.<\/p>\n

There are often many areas of the build process itself that can benefit from some tool that will speed it up. \u00a0Examine your own build and see what is taking the longest amount of time. \u00a0Is there some way to speed that up by developing a tool, or using your build tool in a better way?<\/p>\n

Look carefully at the process developers are using to pull down new code, build code, write code, test code, and check it back in. \u00a0Look at each step in the process. \u00a0Try to find areas that are being repeated, which do not need to be repeated over and over. \u00a0One really good thing to look for is some process that has to be started, followed by a wait, then another thing that has to be done after the wait. <\/strong>When you see this pattern, making the process that has to be done after the wait automatically happens usually has a great return on investment.<\/p>\n

Testing<\/h2>\n

There are huge gains to be made in this area with the proper tools. \u00a0I have talked several times about the automation frameworks I have built<\/a>, but there is much more in this area than just that.<\/p>\n

Consider the creation of test cases. \u00a0Where are they stored? \u00a0How are they written?\u00a0 Closely examining this process normally yields interesting results in finding duplication and wasted effort. \u00a0Copying and pasting rows from spreadsheets takes time and is inefficient. \u00a0If you are going to write and maintain manual tests, then having some tools to help you do that can make a big impact.<\/p>\n

Testing itself can usually be automated. \u00a0Take a good look at what you are testing. \u00a0There are some things that cannot be automated, but usually if you can give something a pass or fail result, it can be automated. \u00a0My best advice here is to use an automation framework, and build a custom application specific framework on top of it.<\/p>\n

Tools to manage where code is deployed and test environments can be very useful depending on the process your organization is using. \u00a0Take a look at the process involved in determining where features exist in what environments to look for areas of optimization. \u00a0Take a look at the process of getting code to different test environments.<\/p>\n

Development<\/h2>\n

Many people don’t think about the automation of code development, but there are often many areas where tools are useful. \u00a0Consider some of the\u00a0commercial\u00a0tools that automatically refactor code or generate UI’s.<\/p>\n

The first place I would start here is looking for any code that developers have to write that is boiler plate code. \u00a0Good examples are code that serializes or deserializes specific objects to or from a location, or any kind of code that is a mimic of some external structure. \u00a0Often tools can be written to generate code once the pattern is well understood. \u00a0Code generation tools are\u00a0extremely\u00a0useful because they are able to reduce the total amount of code that has to be maintained and they save the time of creating that code.<\/p>\n

Unit testing tools or scaffolding can also be useful for setting up basic scenarios or data\u00a0structures\u00a0that often have to be set up in many unit or integration level tests. \u00a0Carefully look at the creation of new unit tests for your code base, and determine where there is common code that is being written to set up data, or mock something out.<\/p>\n

Data<\/h2>\n

There often arises the need to do many things with application data, whether that be transporting it to another environment or\u00a0obfuscating\u00a0it to protect sensitive information, or some other purpose.<\/p>\n

Look carefully at the things you do with data, or what you would like to be able to do with data from you application. \u00a0If you have a DBA that is doing lots of “stuff”\u00a0 you can probably find tools to help him do that “stuff” and bring it down to the teams. \u00a0Relying on DBAs doing “stuff” puts your software development team over a barrel.<\/p>\n

Often teams need to transport portions of data from\u00a0environments to test an issue that is happening in production. \u00a0There are opportunities to create tools that can move the data and change personal information to protect that data.<\/p>\n

The process of altering the database structure is also a good one to consider. \u00a0How do developers make changes to the database structure? \u00a0Is it a long and lengthy process requiring many steps?<\/p>\n

Parting words<\/h2>\n

These are just a few of the areas you should look at when thinking about what kind of tools can be created to help automate your processes. \u00a0The key here is to look for the processes that are being done and to try and find ways to eliminate or automate steps. \u00a0It is not always cost effective to build a tool. \u00a0It depends on the size of the organization, the effort in building the tool, and frequency of use. \u00a0But I have found that there are many opportunities to improve efficiency by automating that are not apparent until you look.<\/p>\n","protected":false},"excerpt":{"rendered":"

Okay, so you’re convinced you need a developer tools team, or at least that development tools are important to build. Now, what kind of tools do you build? There is no answer that works\u00a0universally, otherwise you wouldn’t really need to build anything. \u00a0The key is finding what things your team is doing frequently and automating…<\/p>\n","protected":false},"author":2,"featured_media":541,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_kad_post_transparent":"","_kad_post_title":"","_kad_post_layout":"","_kad_post_sidebar_id":"","_kad_post_content_style":"","_kad_post_vertical_padding":"","_kad_post_feature":"","_kad_post_feature_position":"","_kad_post_header":false,"_kad_post_footer":false,"_kad_post_classname":"","footnotes":""},"categories":[2290,45348,12,1981],"tags":[],"class_list":["post-539","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-architecture","category-frameworks","category-testing","category-tools"],"yoast_head":"\nWhat to Automate - Developer Tools<\/title>\n<meta name=\"description\" content=\"Five areas where developers should consider creating developer tools to automate the development workflow. Details on what specific things to look at.\" \/>\n<meta name=\"robots\" content=\"noindex, follow\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"What to Automate - Developer Tools\" \/>\n<meta property=\"og:description\" content=\"Five areas where developers should consider creating developer tools to automate the development workflow. Details on what specific things to look at.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/simpleprogrammer.com\/what-to-automate\/\" \/>\n<meta property=\"og:site_name\" content=\"Simple Programmer\" \/>\n<meta property=\"article:published_time\" content=\"2010-03-22T17:02:47+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2018-02-23T21:21:13+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/simpleprogrammer.com\/wp-content\/uploads\/2010\/03\/automation.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"325\" \/>\n\t<meta property=\"og:image:height\" content=\"325\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"author\" content=\"John Sonmez\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"John Sonmez\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"6 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\\\/\\\/simpleprogrammer.com\\\/what-to-automate\\\/#article\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/simpleprogrammer.com\\\/what-to-automate\\\/\"},\"author\":{\"name\":\"John Sonmez\",\"@id\":\"https:\\\/\\\/simpleprogrammer.com\\\/#\\\/schema\\\/person\\\/1abc70edfb189184827740310672de67\"},\"headline\":\"What to Automate – Developer Tools\",\"datePublished\":\"2010-03-22T17:02:47+00:00\",\"dateModified\":\"2018-02-23T21:21:13+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\\\/\\\/simpleprogrammer.com\\\/what-to-automate\\\/\"},\"wordCount\":1295,\"commentCount\":1,\"image\":{\"@id\":\"https:\\\/\\\/simpleprogrammer.com\\\/what-to-automate\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/simpleprogrammer.com\\\/wp-content\\\/uploads\\\/2010\\\/03\\\/automation.jpg\",\"articleSection\":[\"Architecture\",\"Frameworks\",\"Testing\",\"Tools\"],\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\\\/\\\/simpleprogrammer.com\\\/what-to-automate\\\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/simpleprogrammer.com\\\/what-to-automate\\\/\",\"url\":\"https:\\\/\\\/simpleprogrammer.com\\\/what-to-automate\\\/\",\"name\":\"What to Automate - Developer Tools\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/simpleprogrammer.com\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/simpleprogrammer.com\\\/what-to-automate\\\/#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/simpleprogrammer.com\\\/what-to-automate\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/simpleprogrammer.com\\\/wp-content\\\/uploads\\\/2010\\\/03\\\/automation.jpg\",\"datePublished\":\"2010-03-22T17:02:47+00:00\",\"dateModified\":\"2018-02-23T21:21:13+00:00\",\"author\":{\"@id\":\"https:\\\/\\\/simpleprogrammer.com\\\/#\\\/schema\\\/person\\\/1abc70edfb189184827740310672de67\"},\"description\":\"Five areas where developers should consider creating developer tools to automate the development workflow. Details on what specific things to look at.\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/simpleprogrammer.com\\\/what-to-automate\\\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/simpleprogrammer.com\\\/what-to-automate\\\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/simpleprogrammer.com\\\/what-to-automate\\\/#primaryimage\",\"url\":\"https:\\\/\\\/simpleprogrammer.com\\\/wp-content\\\/uploads\\\/2010\\\/03\\\/automation.jpg\",\"contentUrl\":\"https:\\\/\\\/simpleprogrammer.com\\\/wp-content\\\/uploads\\\/2010\\\/03\\\/automation.jpg\",\"width\":325,\"height\":325,\"caption\":\"automate\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/simpleprogrammer.com\\\/what-to-automate\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/simpleprogrammer.com\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"What to Automate – Developer Tools\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/simpleprogrammer.com\\\/#website\",\"url\":\"https:\\\/\\\/simpleprogrammer.com\\\/\",\"name\":\"Simple Programmer\",\"description\":\"\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/simpleprogrammer.com\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\\\/\\\/simpleprogrammer.com\\\/#\\\/schema\\\/person\\\/1abc70edfb189184827740310672de67\",\"name\":\"John Sonmez\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/dc2f18dfa76e017566ce607de002d7abe4acfd5ec19a6db82c180b00867b8d94?s=96&d=mm&r=g\",\"url\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/dc2f18dfa76e017566ce607de002d7abe4acfd5ec19a6db82c180b00867b8d94?s=96&d=mm&r=g\",\"contentUrl\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/dc2f18dfa76e017566ce607de002d7abe4acfd5ec19a6db82c180b00867b8d94?s=96&d=mm&r=g\",\"caption\":\"John Sonmez\"},\"url\":\"https:\\\/\\\/simpleprogrammer.com\\\/author\\\/jsonmez\\\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"What to Automate - Developer Tools","description":"Five areas where developers should consider creating developer tools to automate the development workflow. Details on what specific things to look at.","robots":{"index":"noindex","follow":"follow"},"og_locale":"en_US","og_type":"article","og_title":"What to Automate - Developer Tools","og_description":"Five areas where developers should consider creating developer tools to automate the development workflow. Details on what specific things to look at.","og_url":"https:\/\/simpleprogrammer.com\/what-to-automate\/","og_site_name":"Simple Programmer","article_published_time":"2010-03-22T17:02:47+00:00","article_modified_time":"2018-02-23T21:21:13+00:00","og_image":[{"width":325,"height":325,"url":"https:\/\/simpleprogrammer.com\/wp-content\/uploads\/2010\/03\/automation.jpg","type":"image\/jpeg"}],"author":"John Sonmez","twitter_card":"summary_large_image","twitter_misc":{"Written by":"John Sonmez","Est. reading time":"6 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/simpleprogrammer.com\/what-to-automate\/#article","isPartOf":{"@id":"https:\/\/simpleprogrammer.com\/what-to-automate\/"},"author":{"name":"John Sonmez","@id":"https:\/\/simpleprogrammer.com\/#\/schema\/person\/1abc70edfb189184827740310672de67"},"headline":"What to Automate – Developer Tools","datePublished":"2010-03-22T17:02:47+00:00","dateModified":"2018-02-23T21:21:13+00:00","mainEntityOfPage":{"@id":"https:\/\/simpleprogrammer.com\/what-to-automate\/"},"wordCount":1295,"commentCount":1,"image":{"@id":"https:\/\/simpleprogrammer.com\/what-to-automate\/#primaryimage"},"thumbnailUrl":"https:\/\/simpleprogrammer.com\/wp-content\/uploads\/2010\/03\/automation.jpg","articleSection":["Architecture","Frameworks","Testing","Tools"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/simpleprogrammer.com\/what-to-automate\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/simpleprogrammer.com\/what-to-automate\/","url":"https:\/\/simpleprogrammer.com\/what-to-automate\/","name":"What to Automate - Developer Tools","isPartOf":{"@id":"https:\/\/simpleprogrammer.com\/#website"},"primaryImageOfPage":{"@id":"https:\/\/simpleprogrammer.com\/what-to-automate\/#primaryimage"},"image":{"@id":"https:\/\/simpleprogrammer.com\/what-to-automate\/#primaryimage"},"thumbnailUrl":"https:\/\/simpleprogrammer.com\/wp-content\/uploads\/2010\/03\/automation.jpg","datePublished":"2010-03-22T17:02:47+00:00","dateModified":"2018-02-23T21:21:13+00:00","author":{"@id":"https:\/\/simpleprogrammer.com\/#\/schema\/person\/1abc70edfb189184827740310672de67"},"description":"Five areas where developers should consider creating developer tools to automate the development workflow. Details on what specific things to look at.","breadcrumb":{"@id":"https:\/\/simpleprogrammer.com\/what-to-automate\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/simpleprogrammer.com\/what-to-automate\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/simpleprogrammer.com\/what-to-automate\/#primaryimage","url":"https:\/\/simpleprogrammer.com\/wp-content\/uploads\/2010\/03\/automation.jpg","contentUrl":"https:\/\/simpleprogrammer.com\/wp-content\/uploads\/2010\/03\/automation.jpg","width":325,"height":325,"caption":"automate"},{"@type":"BreadcrumbList","@id":"https:\/\/simpleprogrammer.com\/what-to-automate\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/simpleprogrammer.com\/"},{"@type":"ListItem","position":2,"name":"What to Automate – Developer Tools"}]},{"@type":"WebSite","@id":"https:\/\/simpleprogrammer.com\/#website","url":"https:\/\/simpleprogrammer.com\/","name":"Simple Programmer","description":"","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/simpleprogrammer.com\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/simpleprogrammer.com\/#\/schema\/person\/1abc70edfb189184827740310672de67","name":"John Sonmez","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/secure.gravatar.com\/avatar\/dc2f18dfa76e017566ce607de002d7abe4acfd5ec19a6db82c180b00867b8d94?s=96&d=mm&r=g","url":"https:\/\/secure.gravatar.com\/avatar\/dc2f18dfa76e017566ce607de002d7abe4acfd5ec19a6db82c180b00867b8d94?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/dc2f18dfa76e017566ce607de002d7abe4acfd5ec19a6db82c180b00867b8d94?s=96&d=mm&r=g","caption":"John Sonmez"},"url":"https:\/\/simpleprogrammer.com\/author\/jsonmez\/"}]}},"_links":{"self":[{"href":"https:\/\/simpleprogrammer.com\/wp-json\/wp\/v2\/posts\/539","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/simpleprogrammer.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/simpleprogrammer.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/simpleprogrammer.com\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/simpleprogrammer.com\/wp-json\/wp\/v2\/comments?post=539"}],"version-history":[{"count":0,"href":"https:\/\/simpleprogrammer.com\/wp-json\/wp\/v2\/posts\/539\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/simpleprogrammer.com\/wp-json\/wp\/v2\/media\/541"}],"wp:attachment":[{"href":"https:\/\/simpleprogrammer.com\/wp-json\/wp\/v2\/media?parent=539"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/simpleprogrammer.com\/wp-json\/wp\/v2\/categories?post=539"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/simpleprogrammer.com\/wp-json\/wp\/v2\/tags?post=539"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}