Convert Plaster into a DSL #334

Activity

1. TylerLeonhardt commented on Jun 14, 2018

   

   TylerLeonhardt

   on Jun 14, 2018

   Collaborator

   More actions

   This is very interesting. I like the concept. I'm wondering what @rjmholt thinks, he's our DSL expert 😉
2. rkeithhill commented on Jun 14, 2018

   

   rkeithhill

   on Jun 14, 2018

   Collaborator

   More actions

   We went back and forth on this early on and decided on XML because we were sure we could constrain execution using this declarative approach. It wasn't clear to me that a DSL could be constrained. Perhaps that has changed or I was misinformed. Probably the latter.

   The other thing we liked about XML was the schema and being able to version and easily validate manifest files against the schema.

   When I look at this:

   Manifest "Plaster Manifest Template" {
       Metadata {
           Name "NewPlasterTemplate"
           ID "bcd7b44b-0fda-4a08-9857-3e064abd6e2b"
           Version 0.0.1
           Title "A new Plaster Manifest"
           Description "A short description"
           Author "Brent W"
           Tags Tag1, Tag2, Tag3, etc...
       }
       Parameters {
   ...
   

   I don't see much benefit/diff over XML or JSON if you're limiting the manifest to just declarative information without the ability to run arbitrary script. If we decided to punt on constrained execution then a DSL makes more sense IMO.
3. bwright86 commented on Jun 14, 2018

   

   bwright86

   on Jun 14, 2018

   Author

   More actions

   I would agree, if the plan is to stick with only declarative information in the manifest, there would be no real benefit to converting from XML to DSL. On the flip side if the decision is to open up the tool, and allow scripting in the manifest and more familiar scripting in the TemplateFiles, there would be plenty of benefits from changing, including:

   • Readability/Simplicity (Fueling adoption and lessening the learning curve)
   • Extensibility (as people find different use cases for scaffolding)
   • Tooling (There will be a need to do more things that are not in scope for this tool, and the community could provide it)

   I believe it is better to build powerful tools, and provide useful information to educate the user on how to be safe, as well as tooling to inform the user of the quality/safety of community built templates. It would be helpful to have features that can be enabled as the user advances in their knowledge of the tool. So they can start out with a safe area to experiment and learn and do some basic scaffolding, but as the need to build more advanced templates, they can take on the responsibility of safely scaffolding their templates.

   In the end, Plaster is not much different than Powershell, user's can write scripts of varying quality/safety, and they can execute it in their environment. They also have the ability to download scripts from the internet, and it is up to them to decide on the quality/safety of the script before executing it. I believe for Plaster to continue to grow as a scaffolding tool into a fully grown scaffolding system, it will be necessary to move away from a safety-first mentality.

   Thank you for the response.
4. rjmholt commented on Jun 14, 2018

   

   rjmholt

   on Jun 14, 2018

   More actions

   I went to @KevinMarquette's talk at PowerShell Summit actually. It was really interesting and informative.

   The hard part I think is limiting the execution. I'm not really sure how you'd go about fixing that part in current PowerShell. That is an interesting concern though, which is helpful because I keep thinking about how to improve DSL support in PowerShell.

   Even using type constraints and a hashtable body, you still don't prevent arbitrary execution - the RHS of hashtable expressions can be any expression at all, and type constraints will only be applied after the evaluation of an expression.

   The plus side is that you control when the scriptblock executes. So if you can find a way to check the scriptblock for bad things, you can do that check and throw an error if it does. Problem is that that checking is hard and not terribly universal -- you would have to spelunk the AST and do some heuristics, which there are helper methods for, but still. Example:

   $dslBody = {
       Invoke-BadCommand
   }
   
   $exprAsts = $dslBody.Ast.Find({ $args[0] -is [System.Management.Automation.Language.ExpressionAst] }, $true)
   
   Test-BodyExpressionsAreSafe $exprAsts

   A particularly helpful method in that case is $ast.SafeGetValue(), which tries to get a value from the AST node if it is safe to do so.

   Anyway, even in a world with an XML/JSON-based configuration, you can use this quite easily -- you just write a DSL that constructs the XML (this is a pretty rough example, you would want one function for each keyword, and then functions that handle each layer of the XML internally):

   function Manifest
   {
       param([hashtable]$Body)
   
       if ($Body.Metadata)
       {
           $metadata = [System.Xml.Linq.XElement]::new([System.Xml.Linq.XName]"metadata")
   
           $metadataProperties = @{
               name = [string]
               id = [guid]
               version = [version]
               title = [string]
               description = [string]
               author = [string]
               tags = [string[]]
           }
   
           foreach ($metadataProperty in $metadataProperties.Keys)
           {
               if ($Body.Metadata[$metadataProperty])
               {
                   $val = [System.Management.Automation.LanguagePrimitives]::ConvertTo($Body.Metadata[$metadataProperty], $metadataProperties[$metadataProperty])
                   $metadata.Add([System.Xml.Linq.XElement]::new([System.Xml.Linq.XName]$metadataProperty, $val))
               }
           }
       }
   
       $doc = [System.Xml.Linq.XElement]::new([System.Xml.Linq.XName]"plasterManifest", $metadata)
   
       return $doc.ToString()
   }

   This gives you something like this:

   > Manifest @{
   >>   Metadata = @{
   >>       name = "Test Module"
   >>       id = [guid]::NewGuid()
   >>       description = "Something I wrote"
   >>    }
   >> }
   <plasterManifest>
     <metadata>
       <name>Test Module</name>
       <description>Something I wrote</description>
       <id>cc62e080-7788-4f41-8406-efa4b4f1715e</id>
     </metadata>
   </plasterManifest>
   C:\Users\roholt\Documents\Dev\sandbox
   >

   With this model, scriptblocks essentially refer to arrays or unordered bags with possible duplicate keys. So for <parameter/>s, you just stick them in a script block and call & on them.
5. KevinMarquette commented on Jun 14, 2018

   

   KevinMarquette

   on Jun 14, 2018

   More actions

   Just to add to this conversation, there is a community module that already implements this as a DSL. https://github.com/dchristian3188/PlasterManifestDSL @dchristian3188

   👍React with 👍3Reacted by Rob Holt, Keith Hill and TylerLeonhardt
6. rkeithhill commented on Jun 14, 2018

   

   rkeithhill

   on Jun 14, 2018

   Collaborator

   More actions

   I remember a few years back Jason gave a session on DSL ideas/improvements during the MVP summit. I had suggested that we needed some mechanism to restrict what the DSL could execute.

   I think the DSL issue is somewhat orthogonal to the "allow arbitrary/extended execution" request. I think the latter can be solved with an adequately named param like -AllowArbitraryScriptExecution.
7. bwright86 commented on Jun 16, 2018

   

   bwright86

   on Jun 16, 2018

   Author

   More actions

   Thanks @KevinMarquette and @rjmholt for the input, I did not realize there is already a module that adds DSL to Plaster. This would probably work for my needs.

   For this issue, I was looking for a way to extend the Content section of the manifest to allow looping of multiple templates based on the some input.

   Something like this:

   Content {
       foreach ($fruit in $PLASTER_PARAM_FruitSelection) {
           TemplateFile '$fruit.json.txt' 'Fruits\$fruit.json'
       }
   }
   

   But the discussion did lean into #333, where I asked to open up the constrained runspace for more advanced use cases. That issue does have some overlap here, but if something else is creating the manifest without a constrained runspace, I think that would resolve this.

   I will try out the module Kevin M. pointed out, and see if that covers what I was looking for.
   I could be good to close this issue, unless there is more to discuss.

   Thanks again for the responses.

   👍React with 👍1Reacted by TylerLeonhardt
8. TylerLeonhardt commented on Jun 19, 2018

   

   TylerLeonhardt

   on Jun 19, 2018

   Collaborator

   More actions

   I think it would be worthwhile to include a passage in the readme pointing to @dchristian3188's PlasterManifestDSL in case others are looking for a DSL.

   And to talk about how we can make sure that repo keeps up-to-date with any breaking schema changes in Plaster.

   :)

   👍React with 👍1Reacted by Kevin Marquette
9. torgro commented on Oct 5, 2018

   

   torgro

   on Oct 5, 2018

   More actions

   Hi, a bit late to the party, however regarding the arbitrary command execution problem for DSL, would not the scriptblock method CheckRestrictedLanguage solve the issue? The method takes 3 arguments:

   • An IEnummerable collection of allowed variables, typically an array or List.
   • An IEnummerable collection of allowed commands, typically an array or List.
   • A boolean value that indicates if use of Environment variables is allowed ($true makes them allowed)

   /Tore
10. jdhitsolutions

    closed this as completedon Feb 26, 2021
11. PowerShellOrg

    locked and limited conversation to collaborators on Feb 26, 2021